Imaging apparatus, imaging method and computer-readable recording medium

ABSTRACT

An imaging apparatus includes an imaging unit that captures a subject to generate image data of the subject; a display unit that displays the image; an image separating unit that separates a subject image and a background image from the image displayed; a special effect input unit that receives information related to special effects respectively applied to the subject image and the background image; a special effect image generating unit that generates a special effect image of each of the subject and background images by applying special effects corresponding to information received by the special effect input unit; a storing unit that stores special effect information for assigning an advisability corresponding to a combination of special effects which the special effect image generating unit applies to the subject image and the background image; and a synthetic image generating unit that generates a synthetic image using the generated special effect image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2010-198310, filed on Sep. 3, 2010;Japanese Patent Application No. 2010-241321, filed on Oct. 27, 2010,Japanese Patent Application No. 2010-241322, filed on Oct. 27, 2010; andJapanese Patent Application No. 2010-286513, filed on Dec. 22, 2010, theentire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus, an imaging methodand a computer-readable recording medium which capture an image of asubject and generate electronic image data.

2. Description of the Related Art

In recent years, imaging apparatuses such as digital cameras and digitalvideos are known to have a technique which meets various demands ofconsumers by applying special effects to image data obtained bycapturing a subject. For example, a technique is known which can recordshot image data by applying a special effect such as wipe or sepia tothe shot image data (see, for example, Japanese Laid-open PatentPublication No. 2005-347886). This technique allows the user to select adesired special effect by displaying an image to which a special effectselected when the user is operating an effect switch for selecting aspecial effect is applied, on part of an image displayed on the displaymonitor.

Further, an imaging apparatus is known which captures the same subject aplurality of times, separates a subject image and a background imageincluded in an image using a plurality of shot images and blurringprocess of the background image to synthesize the background image withthe subject image again (see, for example, Japanese Patent No. 4321287).

SUMMARY OF THE INVENTION

An imaging apparatus according to an aspect of the present inventionincludes an imaging unit that captures an image of a subject to generateelectronic image data of the subject; a display unit that displays theimage corresponding to the image data; an image separating unit thatseparates a subject image and a background image from the imagedisplayed on the display unit; a special effect input unit that receivesinformation related to special effects respectively applied to thesubject image and the background image separated by the image separatingunit; a special effect image generating unit that generates a specialeffect images for each of the subject image and the background image byapplying special effects corresponding to information received by thespecial effect input unit individually to the subject image and thebackground image; a storing unit that stores special effect informationfor assigning an advisability corresponding to a combination of specialeffects which the special effect image generating unit applies to thesubject image and the background image; a synthetic image generatingunit that generates a synthetic image using the special effect imagegenerated by the special effect image generating unit; and a displaycontrol unit that controls the display unit to display the image. Thespecial effect image generating unit applies a special effect to one ofthe subject image and the background image, and when the synthetic imagegenerating unit synthesizes a special effect image to which the specialeffect is applied, with the other one of the subject image and thebackground image, the display control unit controls the display unit todisplay the synthesized synthetic image, and controls the display unitto display information related to a special effect which needs to be thesubject image or the background image, in order from the highestadvisability corresponding to a combination with a special effectapplied immediately before, referring to special effect informationstored in the storing unit.

An imaging apparatus according to another aspect of the presentinvention includes an imaging unit that captures an image of a subjectto continuously generate electronic image data of the subject; an imagedata storing unit that stores the image data generated by the imagingunit; a display unit that displays the image corresponding to the imagedata generated by the imaging unit; a shooting mode setting unit thatsets one of a plurality of shooting modes for which a frame rate ofcapturing suitable for each capturing scene is set in advance; a specialeffect input unit that receives information related to a special effectfor the image generated at a frame rate of the shooting mode set by theshooting mode setting unit; a special effect image generating unit thatgenerates a special effect image by applying to the image a specialeffect corresponding to information received by the special effect inputunit; an advisability information storing unit that stores advisabilityinformation for assigning an advisability corresponding to a combinationof a special effect which the special effect image generating unitapplies to the image, and the frame rate of the shooting mode; and adisplay control unit that controls the display unit to display the imagesequentially generated at the frame rate of the shooting mode set by theshooting mode setting unit, in a time sequence, and which controls thedisplay unit to display information related to the special effect whichthe special effect image generating unit needs to apply to the image, inorder from the highest advisability corresponding to a combination withthe frame rate of the shooting mode set by the shooting mode settingunit, referring to the advisability information stored in theadvisability information storing unit.

An imaging apparatus according to still another aspect of the presentinvention includes an imaging unit that captures an image of a subjectto generate electronic image data of the subject; a display unit thatcan display the image corresponding to the image data generated by theimaging unit; a shooting mode setting unit that sets one of a pluralityof shooting modes for which a condition of capturing suitable for eachcapturing scene is set in advance; a special effect input unit thatreceives information related to a special effect for the image generatedby the imaging unit according to the shooting mode set by the shootingmode setting unit; a special effect image generating unit that generatesa special effect image by applying to the image a special effectcorresponding to information received by the special effect input unit;an advisability information storing unit that stores advisabilityinformation for assigning an advisability corresponding to a combinationof a special effect which the special effect image generating unitapplies to the image, and the shooting mode; and a display control unitthat controls the display unit to display the image generated by theimaging unit according to the shooting mode set by the shooting modesetting unit, and which controls the display unit to display informationrelated to the special effect which the special effect image generatingunit needs to apply to the image, in order from the highest advisabilitycorresponding to a combination with the shooting mode set by theshooting mode setting unit, referring to the advisability informationstored in the advisability information storing unit.

An imaging method according to still another aspect of the presentinvention is executed by an imaging apparatus that captures an image ofa subject to generate electronic image data of the subject, includes adisplay unit which displays an image corresponding to the generatedimage data, and that can apply a special effect to the image. Theimaging method includes separating a subject image and a backgroundimage from the captured image; generating a special effect image byapplying a special effect to one of the separated subject image andbackground image; generating a synthetic image by synthesizing the otherone of the subject image and the background image, with the specialeffect image; and displaying the synthetic image, reading and referringto special effect information from a storing unit which stores thespecial effect information for assigning an advisability correspondingto a combination of special effects applied to the subject image and thebackground image, and displaying information related to a special effectwhich needs to be applied to the subject image or the background image,in order from the highest advisability corresponding to a combinationwith a special effect applied immediately before.

An imaging method according to still another aspect of the presentinvention is executed by an imaging apparatus that captures an image ofa subject to generate electronic image data of the subject, includes adisplay unit which displays an image corresponding to the generatedimage data, and that can apply a special effect to the image. Theimaging method includes setting in the imaging apparatus one of aplurality of shooting modes for which a frame rate of capturing suitablefor each capturing scene is set in advance; receiving informationrelated to a special effect for the image generated at a frame rate ofthe shooting mode; generating a special effect image by applying to theimage a special effect corresponding to the received information; anddisplaying the image sequentially generated at the frame rate of theshooting mode in a time sequence, reading and referring to advisabilityinformation from an advisability storing unit which stores theadvisability information for assigning an advisability corresponding toa combination of the special effect applied to the image and the framerate of the shooting mode, and displaying information related to thespecial effect which needs to be applied to the image, in order from thehighest advisability corresponding to a combination with the frame rateof the shooting mode.

An imaging method according to still another aspect of the presentinvention is executed by an imaging apparatus that captures an image ofa subject to generate electronic image data of the subject, displays animage corresponding to the generated image data, and can apply a specialeffect to the image. The imaging method includes setting in the imagingapparatus one of a plurality of shooting modes for which a frame rate ofcapturing suitable for each capturing scene is set in advance; receivingan input of information related to a special effect for the imagegenerated at a frame rate of the shooting mode; generating a specialeffect image by applying to the image a special effect corresponding tothe received information; and displaying the image sequentiallygenerated at the frame rate of the shooting mode in a time sequence,reading and referring to advisability information from an advisabilitystoring unit which stores the advisability information for assigning anadvisability corresponding to a combination of the special effectapplied to the image and the frame rate of the shooting mode, anddisplaying information related to the special effect which needs to beapplied to the image, in order from the highest advisabilitycorresponding to a combination with the frame rate of the shooting mode.

A non-transitory computer-readable recording medium according to stillanother aspect of the present invention has an executable program storedthereon. The program instructs a processor of the imaging apparatusaccording to the present invention to perform the imaging methodaccording to the present invention.

The above and other features, advantages and technical and industrialsignificance of this invention will be better understood by reading thefollowing detailed description of presently preferred embodiments of theinvention, when considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagingapparatus according to a first embodiment of the present invention;

FIG. 2 is a perspective view illustrating a configuration of an imagingapparatus according to the first embodiment according to the presentinvention on a front surface side;

FIG. 3 is a perspective view illustrating a configuration of an imagingapparatus according to the first embodiment of the present invention ona back surface side;

FIG. 4 is a schematic view illustrating a situation when an imaging unitincluded in an imaging apparatus according to the first embodiment ofthe present invention generates two items of image data in which oneends of the respective field of views in left and right directionsoverlap;

FIG. 5 is a view illustrating an example of an image corresponding toimage data generated by the imaging unit under the situation illustratedin FIG. 4;

FIG. 6 is a view illustrating an example of an image corresponding toimage data generated by the imaging unit under the situation illustratedin FIG. 4;

FIG. 7 is a view illustrating an example of an image on which theimaging unit virtually overlays images generated at different capturingpositions under the situation illustrated in FIG. 4;

FIG. 8 is a view illustrating an example of a background image generatedby an image separating unit included in an imaging apparatus accordingto the first embodiment of the present invention;

FIG. 9 is a view illustrating an example of a subject image generated byan image separating unit included in an imaging apparatus according tothe first embodiment of the present invention;

FIG. 10 is a schematic view describing an outline of processing of aspecial effect image generating unit and synthetic image generating unitincluded in an imaging apparatus according to the first embodiment ofthe present invention;

FIG. 11 is a flowchart illustrating an outline of processing performedby an imaging apparatus according to the first embodiment of the presentinvention;

FIG. 12 is a flowchart illustrating an outline of menu operation processperformed by an imaging apparatus according to the first embodiment ofthe present invention;

FIG. 13 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the firstembodiment of the present invention;

FIG. 14 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the firstembodiment of the present invention;

FIG. 15 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the firstembodiment of the present invention;

FIG. 16 is a view illustrating an example of a special effectinformation table stored by a special effect information storing unitincluded in an imaging apparatus according to the first embodiment ofthe present invention;

FIG. 17 is a flowchart illustrating an outline of capturing processperformed by an imaging apparatus according to the first embodiment ofthe present invention;

FIG. 18 is a flowchart illustrating an outline of swing art capturingprocess performed by an imaging apparatus according to the firstembodiment of the present invention;

FIG. 19 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the firstembodiment of the present invention;

FIG. 20 is a schematic diagram describing a capturing method of panoramashooting according to a shooting mode performed by an imaging apparatusaccording to the second embodiment of a present invention;

FIG. 21 is a schematic diagram describing a capturing method of HDRshooting according to a shooting mode performed by an imaging apparatusaccording to the second embodiment of the present invention;

FIG. 22 is a view illustrating an example of an image generated by animaging apparatus under the situation illustrated in FIG. 21;

FIG. 23 is a flowchart illustrating an outline of menu operation processperformed by an imaging apparatus according to the second embodiment ofthe present invention;

FIG. 24 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the secondembodiment of the present invention;

FIG. 25 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the secondembodiment of the present invention;

FIG. 26 is a view illustrating an example of a special effectinformation table stored by a special effect information storing unitincluded in an imaging apparatus according to the second embodiment ofthe present invention;

FIG. 27 is a view illustrating an example of an image displayed on adisplay unit included in an imaging apparatus according to the secondembodiment of the present invention;

FIG. 28 is a view illustrating an example of a special effectinformation table stored by a special effect information storing unitincluded in an imaging apparatus according to the second embodiment ofthe present invention;

FIG. 29 is a flowchart illustrating an outline of capturing processperformed by an imaging apparatus according to the second embodiment ofthe present invention;

FIG. 30 is a view illustrating an example of a synthetic image generatedby a synthetic image generating unit included in an imaging apparatusaccording to the second embodiment of the present invention;

FIG. 31 is a block diagram illustrating a configuration of an imagingapparatus according to a third embodiment of the present invention;

FIG. 32 is a flowchart illustrating an outline of processing performedby an imaging apparatus according to the third embodiment of the presentinvention;

FIG. 33 is a view illustrating an example of a situation when an imagingapparatus captures a subject according to a shooting mode, and an imagedisplayed on a display unit;

FIG. 34 is a flowchart illustrating an outline of menu operation processillustrated in FIG. 33;

FIG. 35 is a view illustrating an example of an operation menu displayedon a display unit of an imaging apparatus;

FIG. 36 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 37 is a view illustrating an example of an advisability informationtable stored by an advisability information storing unit of an imagingapparatus;

FIG. 38 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 39 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 40 is a flowchart illustrating an outline of playback/displayprocess illustrated in FIG. 32;

FIG. 41 is a block diagram illustrating a configuration of an imagingapparatus according to a fourth embodiment of the present invention;

FIG. 42 is a flowchart illustrating an outline of processing performedby an imaging apparatus according to the fourth embodiment of thepresent invention;

FIG. 43 is a view illustrating an example of an operation menu displayedon a display unit of an imaging apparatus;

FIG. 44 is a view illustrating an example of a shooting mode menudisplayed on a display unit of an imaging apparatus;

FIG. 45 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 46 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 47 is a view illustrating an example of an advisability informationtable stored by an advisability information storing unit of an imagingapparatus;

FIG. 48 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 49 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 50 is a flowchart illustrating an outline of capturing processillustrated in FIG. 42;

FIG. 51 is a flowchart illustrating an outline of playback/displayprocess illustrated in FIG. 42;

FIG. 52 is a view illustrating an example of a special effect menudisplayed on a display unit of an imaging apparatus;

FIG. 53 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 54 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 55 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus;

FIG. 56 is a view illustrating an example of an image displayed on adisplay unit of an imaging apparatus; and

FIG. 57 is a view describing enlarged display near an area of a touchpanel touched by the user.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

FIG. 1 is view illustrating a configuration of an imaging apparatusaccording to a first embodiment on a side (front surface side) facing asubject. FIG. 2 is a view illustrating a configuration of an imagingapparatus according to the first embodiment on a side (back surfaceside) facing a user. FIG. 3 is a block diagram illustrating aconfiguration of an imaging apparatus according to the first embodimentof the present invention.

As illustrated in FIG. 1 to FIG. 3, an imaging apparatus 1 includes animaging unit 2 which captures a subject and generates electronic imagedata of the shot subject; a posture detecting unit 3 which detects anacceleration of the imaging apparatus 1; a light emitting unit 4 whichradiates light on a field-of-view area shot by the imaging unit 2; aclock 5 which has a decision function of a shooting date and a timerfunction; an operation input unit 6 which receives inputs of variouspieces of information of the imaging apparatus 1; a display unit 7 whichdisplays an image corresponding to image data generated by the imagingunit 2; a touch panel 8 which receives an input of a signalcorresponding to a contact position from an outside; a storing unit 9which stores various pieces of information including image datagenerated by the imaging unit 2; a communication unit 10 which performstransmission and reception to and from various databases through theInternet in mutual directions; and a control unit 11 which controls theoperation of the imaging apparatus 1.

The imaging unit 2 has a lens unit 2 a, a lens driving system 2 b, adiaphragm 2 c, a diaphragm driving system 2 d, a shutter 2 e, a shutterdriving system 2 f, an imaging element 2 g and a signal processing unit2 h.

The lens unit 2 a is formed with, for example, a focus lens or zoomlens, and condenses light from a predetermined field-of-view area. Thelens driving system 2 b is formed with, for example, a DC motor, andmoves, for example, the focus lens or zoom lens of the lens unit 2 a onan optical axis L1 to change, for example, a focus position and focusdistance of the lens unit 2 a.

The diaphragm 2 c adjusts an exposure by limiting an incidence amount oflight condensed by the lens unit 2 a. The diaphragm driving system 2 dhas, for example, a stepping motor, and drives the diaphragm 2 c.

The shutter 2 e sets the state of the imaging element 2 g to an exposurestate or light blocking state. The shutter driving system 2 f has, forexample, a stepping motor, and drives the shutter 2 e according to arelease signal.

The imaging element 2 g is realized by, for example, CCD (Charge CoupledDevice) or CMOS (Complementary Metal Oxide Semiconductor) which receivesand converts light condensed by the lens unit 2 a into an electricalsignal (analog signal), and outputs the converted electrical signal tothe signal processing unit 2 h.

The signal processing unit 2 h applies signal processing such asamplification to the electrical signal outputted from the imagingelement 2 g, and then A/D converts the electrical signal into digitalimage data to output to the control unit 11.

The posture detecting unit 3 includes an acceleration sensor, anddetects a posture state and/or motion of the imaging apparatus 1 bydetecting the acceleration of the imaging apparatus 1. Morespecifically, the posture detecting unit 3 detects the posture of theimaging apparatus 1 based on the horizontal plane.

The light emitting unit 4 has, for example, a xenon lamp or LED (LightEmitting Diode). The light emitting unit 4 radiates strobe light ofauxiliary light toward the field-of-view area of the imaging unit 2.

The clock 5 generates a time signal based on which the imaging apparatus1 operates. By this means, the control unit 11 can set, for example, anacquisition time of image data and an exposure time of the imagingelement 2 g.

The operation input unit 6 includes a power switch 6 a which switches apower state of the imaging apparatus 1 to an on state or off state; arelease switch 6 b which inputs a release signal for commanding tocapture an image; a mode switch 6 c which switches between variousshooting modes of the imaging apparatus 1; a special effect input switch6 d which switches between various settings of the imaging apparatus 1and receives an input of information related to a special effect; a menuswitch 6 e which makes the display unit 7 display information related tothe various shooting modes of the imaging apparatus 1; and a zoom switch6 f which performs a zoom operation of the imaging unit 2.

The display unit 7 is formed using a display panel including, forexample, liquid crystal or organic EL (Electro Luminescence). Thedisplay unit 7 displays an image corresponding to image data generatedby the imaging unit 2. The display unit 7 adequately displays operationinformation of the imaging apparatus 1 and information related toshooting. Further, the display unit 7 displays a three-dimensional image(hereinafter, “3D image”) corresponding to image data generated by theimaging unit 2. Generally, there is a parallax barrier scheme orlenticular lens scheme adopted in a display unit which displays a 3Dimage. With the first embodiment, both schemes are applicable to thedisplay unit.

The touch panel 8 is overlaid and provided on a display screen of thedisplay unit 7 (see FIG. 2). The touch panel 8 detects the positionwhich the user contacts (touches) based on information displayed on thedisplay unit 7, and receives an input of the operation signalcorresponding to this contact position. Generally, there are a resistivetype, a capacitance type and an optical type for a touch panel. With thepresent embodiment, all types are applicable to the touch panel. Inaddition, with the first embodiment, the touch panel 8 functions as oneof the operation input unit 6.

The storing unit 9 includes an image data storing unit 9 a which storesimage data captured by the imaging unit 2; a program storing unit 9 bwhich stores various programs executed by the imaging apparatus 1 andcapturing programs according to the first embodiment; a temporary imagedata storing unit 9 c which temporarily stores image data captured bythe imaging unit 2 or various settings of the imaging apparatus 1; and aspecial effect information storing unit 9 d which stores a specialeffect information table which assigns an advisability corresponding toa combination of special effects applied to a subject image andbackground image when special effects are applied. The storing unit 9 isprovided inside the imaging apparatus 1 in a fixed manner, and is formedusing a semiconductor memory such as a flash memory or RAM (RandomAccess Memory). In addition, the storing unit 9 may have a function of astorage medium interface which reads information stored in the storagemedium while storing the information in a storage medium such as amemory card attached from an outside.

The communication unit 10 has a function of the communication interface,and performs transmission and reception to and from an externalprocessing device such as a server (not illustrated) or personalcomputer through a network (not illustrated) in a mutual direction. Thecommunication unit 10 acquires various items of program data of theimaging apparatus 1 and data related to a special effect informationtable by performing transmission and reception to and from the externalprocessing device. In addition, the communication unit 10 is connectedto the network through a cable or wireless LAN (Local Area Network).

The control unit 11 is formed with, for example, a CPU (CentralProcessing Unit). The control unit 11 reads a program from the programstoring unit 9 b of the storing unit 9 according to, for example, anoperation signal from the operation input unit 6 to execute, andperforms overall control of the operation of the imaging apparatus 1 by,for example, commanding each unit forming the imaging apparatus 1 ortransferring data. The control unit 11 has an image processing unit 11a, an image separating unit 11 b, a special effect image generating unit11 c, a synthetic image generating unit 11 d and a display control unit11 e.

The image processing unit 11 a applies various image processings toimage data outputted from the signal processing unit 2 h to output tothe storing unit 9. More specifically, the image processing unit 11 aapplies processings such as edge enhancement, color correction and γcorrection to image data outputted from the signal processing unit 2 h.

The image separating unit 11 b separates a subject image and backgroundimage from an image corresponding to image data generated by the imagingunit 2. More specifically, using the principle of common triangulation,the image separating unit 11 b separates the subject image andbackground image, based on a parallax of two images corresponding to twoitems of data which are generated by the imaging unit 2 and in which oneends of respective field of views in left and right directions overlapeach other.

The special effect image generating unit 11 c generates a special effectimage of each image by individually applying special effectscorresponding to information received by the touch panel 8 as input, tothe subject image and background image separated by the image separatingunit 11 b. More specifically, for example, the special effect imagegenerating unit 11 c applies, to image data, processing including one ofa plurality of image processings, tone curve processing, blurringprocess, shading adding process, image synthesizing process, noisesuperimposing process, and chroma adjusting process. In addition, thespecial effect image generating unit 11 c may generate a special effectimage by applying a special effect to one image corresponding to imagedata generated by the imaging unit 2. Further, the special effect imagegenerating unit 11 c may continuously generate special effect images bysequentially applying special effects to image data continuouslygenerated by the imaging unit 2.

The synthetic image generating unit 11 d generates a synthetic imageusing the special effect image generated by the special effect imagegenerating unit 11 c. More specifically, the synthetic image generatingunit 11 d generates a synthetic image by synthesizing the subject imageseparated by the image separating unit 11 b with the special effectimage generated by applying a special effect to the background image bythe special effect image generating unit 11 c.

The display control unit 11 e controls the display unit 7 to performdisplay. When the special effect image generating unit 11 c applies aspecial effect to one of the subject image and background image, and thesynthetic image generating unit 11 d synthesizes the special effectimage to which this special effect is applied, with the other one of thesubject image and background image, the display control unit 11 econtrols the display unit 7 to display this synthesized synthetic image,and controls the display unit 7 to display information related to thespecial effect which needs to be applied to the subject image orbackground image, in order from the highest advisability correspondingto the combination with a special effect applied immediately before,referring to the special effect information table stored in the specialeffect information storing unit 9 d. The display control unit 11 econtrols the display unit 7 to display information related to thespecial effects by means of icons. More specifically, the displaycontrol unit 11 e controls the display unit 7 to display a specialeffect icon corresponding to special effect information in order fromthe highest advisability corresponding to the combination with thespecial effect applied immediately before, referring to the specialeffect information table.

A situation will be described where, in the imaging apparatus 1employing the above configuration, the image separating unit 11 bseparates an image generated by the imaging unit 2, into a subject imageand background image. FIG. 4 is a schematic view illustrating asituation when the imaging unit 2 generates two items of image data inwhich one ends of the respective field of views in left and rightdirections overlap. FIG. 5 is a view illustrating an example of an imagecorresponding to image data generated by the imaging unit 2 at acapturing position P1 under the situation illustrated in FIG. 4. FIG. 6is a view illustrating an example of an image corresponding to imagedata generated by the imaging unit 2 at a capturing position P2 underthe situation illustrated in FIG. 4. FIG. 7 is a view illustrating anexample of an image on which the imaging unit 2 virtually overlaysimages generated at different capturing positions under the situationillustrated in FIG. 4. In (a) of FIG. 7, an image W1 is obtained byoverlaying images based on the background. Further, in (b) of FIG. 7, animage W2 is obtained by overlaying images based on the subject. Inaddition, in FIG. 4, the user moves the imaging apparatus 1 from theleft side to the right side, and the imaging apparatus 1 continuouslycaptures images in very short time intervals during this period ofmovement. Hence, there are a plurality of images between the image WL1and image WR1 illustrated in FIG. 5 and FIG. 6.

As illustrated in FIG. 4, the imaging unit 2 continuously generates aplurality of items of image data having different field-of-view areas bycapturing images of a subject A1 (distance d1) and a subject A2(distance d2) having different distances from the imaging unit 2, at aplurality of capturing positions. More specifically, as illustrated inFIG. 5 and FIG. 6, the imaging unit 2 generates the image WL1 at thecapturing position P1 in FIG. 4, and generates the image WR1 at thecapturing position P2 in FIG. 4. As illustrated in FIG. 4, the distanced2 between the imaging unit 2 and subject A2 is longer than the distanced1 between the imaging unit 2 and subject A1, so that the areas of thesubject A2 of the background nearly overlap. More specifically, asillustrated in (b) of FIG. 7, the areas of the subject A2 nearly overlapin the image W2. By contrast with this, as illustrated in (a) of FIG. 7,in the image W1, the areas of the subject A1 do not overlap, and thereis a parallax a1 with respect of the subject A1.

Thus, as illustrated in (a) of FIG. 7, the image separating unit 11 bspecifies areas which overlap when the image WL1 and image WR1 arevirtually overlaid, as a background image. Further, as illustrated in(b) of FIG. 7, the image separating unit 11 b specifies an overlappingarea as a subject image of the subject A1 by overlaying the image WL1and image WR1 based on areas of the image WL1 and image WR1 having highbrightnesses while moving the image WL1 and image WR1 in differentdirections.

By this means, as illustrated in FIG. 8 and FIG. 9, the image separatingunit 11 b can separate a background image W3 and a subject image W4 fromthe image corresponding to image data generated by the imaging unit 2.In addition, when the image WL1 and image WR1 virtually overlap based onthe subject A1, the image separating unit 11 b may specify a subjectimage by overlaying the image WL1 and image WR1 based on areas of theimage WL1 and image WR1 having high contrasts.

Next, an outline of processings of the special effect image generatingunit 11 c and the synthetic image generating unit 11 d will bedescribed. FIG. 10 is a schematic view describing the outline of theprocessings of the special effect image generating unit 11 c and thesynthetic image generating unit 11 d.

As illustrated in FIG. 10, the special effect image generating unit 11 cfirst applies a special effect set by the user such as “soft effect”(described below) to the background image W10 stored in the image datastoring unit 9 a to generate a special effect image W11 ((a)→(b) of FIG.10). Then, the synthetic image generating unit 11 d synthesizes thesubject image W4 separated by the image separating unit 11 b, with thespecial effect image W11 to generate a synthetic image W12 ((b)→(c) ofFIG. 10). Finally, the special effect image generating unit 11 c appliesa special effect set by the user such as “sepia effect” (describedbelow) to the subject image W4 included in the synthetic image W12 togenerate a synthetic image W13 ((c)→(d) of FIG. 10). By this means, theimaging apparatus 1 can generate a synthetic image obtained by applyingspecial effects to the subject image and background image. In addition,in (a) to (d) of FIG. 10, the image to which “soft effect” is applied isgradation processed, and the image to which “sepia effect” is applied isshown with hatching.

Next, processing performed by the imaging apparatus 1 according to thefirst embodiment will be described. FIG. 11 is a flowchart illustratingan outline of processing performed by the imaging apparatus 1.

In FIG. 11, the control unit 11 first decides whether or not the powersource of the imaging apparatus 1 is turned on (step S101). When thepower source of the imaging apparatus 1 is turned on (step S101: Yes),the imaging apparatus 1 goes to step S102. By contrast with this, whenthe power source of the imaging apparatus 1 is not turned on (step S101:No), the imaging apparatus 1 finishes the processing.

Subsequently, the control unit 11 decides whether or not the imagingapparatus 1 is set to a shooting mode (step S102). When the imagingapparatus 1 is set to the shooting mode (step S102: Yes), the imagingapparatus 1 goes to step S103 (described below). By contrast with this,when the imaging apparatus 1 is not set to the shooting mode (step S102:No), the imaging apparatus 1 goes to step S108 (described below).

A case (step S102: Yes) will be described where, in step S102, theimaging apparatus 1 is set to the shooting mode. In this case, thedisplay control unit 11 e controls the display unit 7 to display liveview images corresponding to image data continuously generated by theimaging unit 2 at constant and very small time intervals (step S103).

Subsequently, the control unit 11 decides whether or not a releasesignal for commanding to capture an image is inputted when the useroperates the release switch 6 b (step S104). When a release signal forcommanding to capture an image is not inputted (step S104: No), thecontrol unit 11 decides whether or not the user operates the menu switch6 e (step S105). When the user does not operate the menu switch 6 ewithin a predetermined time, for example, within 30 seconds (step S105:No), the imaging apparatus 1 returns to step S101. By contrast withthis, when the user operates the menu switch 6 e (step S105: Yes), theimaging apparatus 1 executes menu operation process of setting a menuincluded in the shooting mode (step S106), and returns to step S101.

A case (step S104: Yes) will be described where, in step S104, a releasesignal for commanding to capture an image is inputted. In this case, theimaging apparatus 1 executes a process of capturing a subject (stepS107), and returns to step S101.

A case (step S102: No) will be described where, in step S102, theimaging apparatus 1 is not set to the shooting mode. In this case, thedisplay control unit 11 e controls the display unit 7 to display animage selection screen which collectively displays a plurality of imagesstored in the image data storing unit 9 a (step S108).

Subsequently, the control unit 11 decides whether or not the usertouches the touch panel 8 to select an image on the image selectionscreen displayed on the display unit 7 (step S109). When the user doesnot select an image on the image selection screen (step S109: No), theimaging apparatus 1 goes to step S113 (described below). By contrastwith this, when the user selects an image on the image selection screen(step S109: Yes), the display control unit 11 e controls the displayunit 7 to display the image selected by the user on the full screen(step S110), and the imaging apparatus 1 goes to step S111.

In step S111, whether or not the user performs an operation to switchthe image is decided. When the user performs the operation to switch theimage (step S111: Yes), the display control unit 11 e switches the imagecurrently displayed on the display unit 7 (step S112), and the imagingapparatus 1 returns to step S110. By contrast with this, when the userdoes not perform the operation to switch the image (step S111: No), thecontrol unit 11 decides whether or not an operation to playback theimage is performed (step S113). More specifically, the control unit 11decides whether or not the imaging apparatus 1 is switched from aplayback mode to the shooting mode when the user operates the modeswitch 6 c. When the user does not perform an operation to finishplaying an image (step S113: No), the imaging apparatus 1 returns tostep S108. By contrast with this, when the user performs the operationto finish playing back an image (step S113: Yes), the imaging apparatus1 returns to step S101.

Next, menu operation process in step S106 in FIG. 11 will be described.FIG. 12 is a flowchart illustrating an outline of menu operationprocess.

In FIG. 12, the display control unit 11 e first controls the displayunit 7 to display menu icons on the image displayed on the display unit7 (step S201). More specifically, as illustrated in FIG. 13, the displaycontrol unit 11 e displays a shooting mode icon Q1, a strobe light iconQ2, a special shooting mode icon Q3 and a back icon Q4 on an image W21displayed on the display unit 7.

The shooting mode icon Q1 receives an input of a command signal forcommanding the display unit 7 to display detailed parameters of anoperation condition of the shooting mode. As the detailed parameters ofthe image capturing operation condition, parameters such as a shutterspeed, exposure correction range, ISO sensitivity and aperture value aredisplayed.

The strobe light icon Q2 receives an input of a command signal forcommanding the display unit 7 to display detailed parameters of a strobelight operation condition of strobe light (flash). As the detailedparameters of the strobe light operation condition, parameters such asautomatic light emission, light emission prohibition, red-eye lightemission and forced light emission are displayed.

The special shooting mode icon Q3 receives an input of a command signalfor commanding the display unit 7 to display a submenu of the specialshooting mode.

Subsequently, the control unit 11 decides whether or not the useroperates the special shooting mode icon Q3 (step S202). Morespecifically, as illustrated in FIG. 13, when the user touches thespecial shooting mode icon Q3 displayed on the image W21 on the displayunit 7, the control unit 11 decides whether or not the special shootingmode icon Q3 is operated according to the signal received by the touchpanel 8 as input. When the user does not operate the special shootingmode icon Q3 within a predetermined time, for example, within 30 seconds(step S202: No), the imaging apparatus 1 returns to a main routineillustrated in FIG. 11. By contrast with this, when the user operatesthe special shooting mode icon Q3 (step S202: Yes), the imagingapparatus 1 goes to step S203.

Then, the display control unit 11 e displays submenu icons of thespecial shooting mode, on the image displayed on the display unit 7(step S203). More specifically, as illustrated in FIG. 14, the displaycontrol unit 11 e displays a swing synthesis icon Q5, an art filter iconQ6, a swing art icon Q7 and the back icon Q4, on the image W21 displayedon the display unit 7.

Hereinafter, various icons illustrated in FIG. 14 will be described. Theswing synthesis icon Q5 receives an input of a command signal forsetting the shooting mode of the imaging apparatus 1 to a swingsynthesis shooting mode. The swing synthesis shooting mode refers to ashooting mode of generating a synthetic image by synthesizing one of asubject image or background image separated from an image captured bythe image separating unit 11 b, with the other image. The art filtericon Q6 receives an input of a command signal for setting the shootingmode of the imaging apparatus 1 to special effect capturing. The specialeffect capturing refers to a shooting mode of generating a specialeffect image by applying to the captured image a special effect set inthe special effect image generating unit 11 c by the user. The swing articon Q7 receives an input of a command signal for setting the shootingmode of the imaging apparatus 1 to a swing art shooting mode. The swingart shooting mode refers to a shooting mode of generating a specialeffect image by applying a special effect set in the special effectimage generating unit 11 c by the user, to one of a subject image orbackground image separated from the image captured by the imageseparating unit 11 b, and synthesizing this generated special effectimage with the other image and generating a synthetic image.

Referring back to FIG. 12, in step S204, the control unit 11 decideswhether or not the user operates the swing art icon Q7. Morespecifically, as illustrated in FIG. 14, when the user touches the swingart icon Q7 displayed on the image W21 on the display unit 7, thecontrol unit 11 decides whether or not the swing art icon Q7 is operatedaccording to the signal received by the touch panel 8 as input. When theuser does not operate the swing art icon Q7 within a predetermined time(step S204: No), the imaging apparatus 1 returns to the main routineillustrated in FIG. 11. By contrast with this, when the user operatesthe swing art icon Q7 (step S204: Yes), the imaging apparatus 1 goes tostep S205.

Subsequently, the display control unit 11 e controls the display unit 7to display a background image corresponding to image data stored in theimage data storing unit 9 a and a plurality of special effect icons(step S205). More specifically, as illustrated in FIG. 15, the displaycontrol unit 11 e controls the display unit 7 to display the backgroundimage W22, and controls the display unit 7 to display a normal icon Q8,a soft icon Q9 and a sepia icon Q10 referring to a special effectinformation table stored in the special effect information storing unit9 d.

Meanwhile, a method of displaying special effect icons which the displaycontrol unit 11 e controls the display unit 7 to display will bedescribed with reference to FIG. 16. FIG. 16 is a view illustrating anexample of the special effect information table stored by the specialeffect information storing unit 9 d.

As illustrated in FIG. 16, in a special effect information table T1,special effects are written which are applicable to a subject image andbackground image. More specifically, the special effects which areapplicable to the subject image include “normal”, “pop”, “fish-eye”,“soft”, “sepia”, “black/white”, “sketch” and “miniature”. By contrastwith this, the special effects which are applicable to the backgroundinclude “normal”, “pop”, “fish-eye”, “soft”, “sepia”, “black/white”,“sketch” and “miniature”.

Further, in the special effect information table T1, (0 to 5) arewritten for advisabilities which match combinations of special effectsto generate a synthetic image by applying special effects to a subjectimage and background image. More specifically, when the special effectfor the subject image is “normal”, the advisability is “5” for thecombination with a special effect of “normal”, “soft” and “sepia” forthe background. Further, the advisability is “4” for a combination with“pop”, “black/white” and “sketch”, and the advisability is “1” for acombination with “fish-eye” and “miniature”.

That is, a higher numerical value of the advisability is set for abetter combination of special effects. In addition, the numerical valuesillustrated in FIG. 16 are only examples and by no means limited tothese, and a numerical value of the advisability may be changed by, forexample, the special effect input switch 6 d. Further, a lower numericalvalue of the advisability may be set for a better combination of specialeffects.

Thus, the display control unit 11 e controls the display unit 7 todisplay special effect icons in order from the highest advisability of acombination with a special effect, referring to the special effectinformation table T1. For example, as illustrated in FIG. 15, when aspecial effect is not applied to the background image W22 currentlydisplayed on the display unit 7 and the state of the background imageW22 is “normal”, the display control unit 11 e controls the display unit7 to display the normal icon Q8, soft icon Q9 and sepia icon Q10 asspecial effect icons. By this means, the imaging apparatus 1 canpresent, to the user, information which serves as a guide to select anoptimal combination of special effects for the image currently displayedon the display unit 7.

Next, the special effects illustrated in FIG. 16 will be describedrespectively.

Normal refers to the state where the special effect image generatingunit 11 c applies no special effect.

Pop provides an effect of emphasizing a color impressively andcolorfully. For example, processing of improving the hue and saturationof image data is performed.

Fish-eye provides an effect of finishing an image by distorting a linein an arc shape as if the line is captured using a fish-eye lens. Forexample, processing of changing image data to an image distorted by, forexample, coordinate conversion is performed.

Soft provides an effect of finishing the entire image softly as if theimage was captured using a soft focus lens. For example, processing ofdecreasing levels of contrast, gradation, synthesis and saturation ofimage data is performed.

Sepia provides an effect of finishing a sepia and monochrome image. Forexample, processing of decreasing levels of the white balance and tonecurve is performed.

Black/white provides an effect of finishing a black and white imagerepresented by a gray tone. For example, processing of decreasing thelevel of the white balance is performed.

Sketch provides an effect of finishing a sketchy image in which theprofile is drawn by line drawing.

Miniature provides an effect of adjusting focus on the center of thescreen and decreasing levels of contrast and blurring process in thesurrounding such as upper and lower parts. More specifically, withminiature, shading adding process and image synthesizing process areperformed.

Referring back to FIG. 12, in step S206, the control unit 11 decideswhether or not the user performs an operation to switch the backgroundimage. More specifically, as illustrated in FIG. 15, the control unit 11decides whether or not the user touches the image switch icon Q11 whichreceives an input of a command signal for commanding to switch toanother background image. When the user performs the operation to switchthe background image (step S206: Yes), the display control unit 11 econtrols the display unit 7 to switch the background image currentlydisplayed on the display unit 7 to another background image stored inthe image data storing unit 9 a to display on the display unit 7 (stepS207), and the imaging apparatus 1 returns to step S205. By contrastwith this, when the user does not perform an operation to switch thebackground image (step S206: No), the imaging apparatus 1 goes to stepS208.

Subsequently, the control unit 11 decides whether or not the useroperates a special effect icon (step S208). More specifically, asillustrated in FIG. 15, the control unit 11 decides whether or not theuser touches one of the normal icon Q8, soft icon Q9 and sepia icon Q10displayed on the screen of the display unit 7. When the user operates aspecial effect icon (step S208: Yes), the special effect imagegenerating unit 11 c generates a special effect image by applying aspecial effect such as soft corresponding to the special effect icontouched by the user, to the background image currently displayed on thedisplay unit 7 (step S209), and the display control unit 11 e controlsthe display unit 7 to display the special effect image (see, forexample, (b) of FIG. 10) (step S210).

Then, the control unit 11 controls the temporary image data storing unit9 c to temporarily store the special effect image currently displayed onthe display unit 7 (step S211), and the imaging apparatus 1 returns tothe main routine illustrated in FIG. 11.

A case (step S208: No) will be described where, in step S208, the userdoes not operate a special effect icon. In this case, the control unit11 decides whether or not the user performs an operation to switch aspecial effect icon within a predetermined time (step S212). Morespecifically, as illustrated in FIG. 15, the control unit 11 decideswhether or not the user touches the switch icon Q12 which receives aninput of a command signal for commanding to switch to a special effecticon of the second highest advisability from the special effect iconcurrently displayed on the display unit 7. When the user does notperform the operation to switch a special effect icon within apredetermined period, for example, within 30 seconds (step S212: No),the imaging apparatus 1 returns to the main routine illustrated in FIG.11. By contrast with this, when the user performs the operation toswitch a special effect icon (step S212: Yes), the display control unit11 e controls the display unit 7 to switch the special effect icon to aspecial effect icon of the second highest advisability such as “pop”,“black/white” or “sketch” to display, referring to the special effectinformation table T1 stored in the special effect information storingunit 9 d (step S213), and the imaging apparatus 1 returns to step S205.

Next, capturing process in step S107 in FIG. 11 will be described. FIG.17 is a flowchart illustrating an outline of capturing process. Inaddition, a case will be described below where the shooting mode of theimaging apparatus 1 is set to swing art shooting.

As illustrated in FIG. 17, the control unit 11 first decides whether ornot the shooting mode of the imaging apparatus 1 is set to swing artshooting (step S301). When the shooting mode of the imaging apparatus 1is set to swing art shooting (step S301: Yes), the imaging apparatus 1executes swing art capturing process of generating a synthetic image byapplying special effects to a subject image and background image (stepS302), and the imaging apparatus 1 returns to the main routineillustrated in FIG. 11.

A case (step S301: No) will be described where, in step S301, theshooting mode of the imaging apparatus 1 is not set to swing artshooting. In this case, the imaging apparatus 1 captures an image of thesubject (step S303), and the display control unit 11 e controls thedisplay unit 7 to REC-view display an image corresponding to thecaptured image data (step S304).

Subsequently, the control unit 11 controls the image data storing unit 9a of the storing unit 9 to store the captured image data (step S305),and the imaging apparatus 1 returns to the main routine illustrated inFIG. 11. In addition, in step S305, compressed data (JPEG) anduncompressed data (RAW) may be stored together as image data. In thiscase, upon image synthesis described below, the user may select one ofthese items of image data. Further, the user may set various types ofaspect ratios such as 4:3 and 16:9 for the background image, and selecta desired aspect ratio depending on expression or the purpose.

Next, swing art capturing process in step S302 in FIG. 17 will bedescribed. FIG. 18 is a flowchart illustrating an outline of swing artcapturing process.

In FIG. 18, the control unit 11 first decides whether or not the userstarts moving the imaging apparatus 1 (step S401). More specifically,when the user moves the imaging apparatus 1, the control unit 11 decideswhether or not the imaging apparatus 1 starts moving, based on theacceleration detected by the posture detecting unit 3 (see FIG. 4). Whenthe imaging apparatus 1 does not start moving (step S401: No), thecontrol unit 11 repeats this decision. By contrast with this, when theimaging apparatus 1 starts moving (step S401: Yes), the imagingapparatus 1 goes to step S402.

Subsequently, the imaging apparatus 1 continuously captures the subjectin very short intervals (step S402), and the control unit 11 controlsthe temporary image data storing unit 9 c of the storing unit 9 totemporarily store the captured images (step S403).

Then, the control unit 11 decides whether or not the imaging apparatus 1stops moving (step S404). When the imaging apparatus 1 does not stopmoving (step S404: No), the imaging apparatus 1 returns to step S402. Bycontrast with this, when the imaging apparatus 1 stops moving (stepS404: Yes), the imaging apparatus 1 goes to step S405.

Subsequently, the control unit 11 decides whether or not the imageseparating unit 11 b can separate a subject image and background imagefrom a plurality of captured images (step S405). More specifically, thecontrol unit 11 decides whether or not the image separating unit 11 bcan separate a subject image and background image from the capturedimages, based on the parallax of the subject included in a plurality ofimages, for example, adjacent images stored in the temporary image datastoring unit 9 c (see FIG. 7). When the image separating unit 11 bcannot separate the subject image and background image from the capturedimages (step S405: No), the display control unit 11 e controls thedisplay unit 7 to display a warning that the subject image andbackground image cannot be separated from the captured images (stepS406), and the imaging apparatus 1 returns to the main routineillustrated in FIG. 17.

A case (step S405: Yes) will be described where, in step S405, the imageseparating unit 11 b can separate the subject image and background imagefrom the captured images. In this case, the image separating unit 11 bseparates the subject image from the captured images (step S407), andthe synthetic image generating unit 11 d generates a synthetic image bysynthesizing the subject image separated by the image separating unit 11b with a special effect image generated by the special effect imagegenerating unit 11 c (see, for example, (c) of FIG. 10) (step S408).

Subsequently, the display control unit 11 e controls the display unit 7to display the synthetic image generated by the synthetic imagegenerating unit 11 d (step S409), and the control unit 11 decideswhether or not the user operates the menu switch 6 e (step S410). Whenthe user operates the menu switch 6 e (step S410: Yes), the imagingapparatus 1 goes to step S411 (described below). By contrast with this,when the user does not operate the menu switch 6 e within apredetermined time, for example, within 30 seconds (step S410: No), theimaging apparatus 1 goes to step S419 (described below).

A case (step S410: Yes) will be described where, in step S410, the useroperates the menu switch 6 e. In this case, the display control unit 11e controls the display unit 7 to display an image selection icon andspecial effect icons on an image displayed on the display unit 7 (stepS411). More specifically, as illustrated in FIG. 19, the display controlunit 11 e controls the display unit 7 to display a background selectionicon Q13 for commanding to switch to a special effect icon of a specialeffect applied to the background image, on the synthetic image W23displayed on the display unit 7. Further, referring to the specialeffect information table T1, the display control unit 11 e controls thedisplay unit 7 to display information related to a special effect whichneeds to be applied to the subject image, in order from the highestadvisability corresponding to the combination with a special effectapplied immediately before. For example, the display control unit 11 econtrols the display unit 7 to display a normal icon Q14 and a soft iconQ15 when the special effect which the special effect image generatingunit 11 c applies immediately before is “soft”. Further, the displaycontrol unit 11 e controls the display unit 7 to display a switch iconQ16 for receiving an input of a command signal for switching a specialeffect icon, and an end icon Q17 for receiving an input of a commandsignal for finishing swing art shooting.

Subsequently, the control unit 11 decides whether or not the useroperates the image selection icon (step S412). More specifically, asillustrated in FIG. 19, the control unit 11 decides whether or not theuser operates the background selection icon Q13. When the user operatesthe image selection icon (step S412: Yes), the display control unit 11 econtrols the display unit 7 to switch and display the image selectionicon and special effect icons (step S413), and the imaging apparatus 1returns to step S412. More specifically, the display control unit 11 econtrols the display unit 7 to display a subject icon (not illustrated)as the image selection icon, on the synthetic image W23 displayed on thedisplay unit 7, and to display information related to a special effectwhich needs to be applied to the background image in order from thehighest advisability corresponding to a combination with the specialeffect applied immediately before, referring to the special effectinformation table T1.

A case (step S412: No) will be described where, in step S412, the userdoes not operate the image selection icon. In this case, the controlunit 11 decides whether or not the user operates the special effect icon(step S414). When the user operates the special effect icon (step S414:Yes), the special effect image generating unit 11 c generates a specialeffect image by applying to the subject image a special effectcorresponding to the special effect icon which the touch panel 8receives as input in a contact area positioned on the display area ofthe special effect icon (step S415).

Subsequently, the synthetic image generating unit 11 d generates asynthetic image using the special effect image generated by the specialeffect image generating unit 11 c (step S416), the display control unit11 e controls the display unit 7 to display the synthetic imagegenerated by the synthetic image generating unit 11 d (step S417).

Then, the control unit 11 decides whether or not the user operates theend icon Q17 within a predetermined time (step S418). When the user doesnot operate the end icon Q17 within a predetermined time, for example,within 30 seconds (step S418: No), the imaging apparatus 1 returns tostep S412. By contrast with this, when the user operates the end iconQ17 within a predetermined time (step S418: Yes), the control unit 11controls the image data storing unit 9 a to store image datacorresponding to image currently displayed on the display unit 7 (stepS419), and the imaging apparatus 1 returns to the main routineillustrated in FIG. 17.

A case (step S414: No) will be described where, in step S414, the userdoes not operate the special effect icon. In this case, the control unit11 decides whether or not the user performs an operation to switch aspecial effect icon (step S420). More specifically, the control unit 11decides whether or not the user operates the switch icon Q16 (see FIG.19). When the user does not perform an operation to switch the specialeffect icon (step S420: No), the imaging apparatus 1 goes to step S418.By contrast with this, when the user performs an operation to switch thespecial effect icon (step S420: Yes), the display control unit 11 econtrols the display unit 7 to switch the special effect icon to aspecial effect icon of the second highest advisability to display,referring to the special effect information table T1 (step S421), andthe imaging apparatus 1 returns to step S412. More specifically, thedisplay control unit 11 e controls the display unit 7 to display specialeffect icons “sepia” and “black/white” as the special effect icons ofthe second highest advisabilities.

According to the first embodiment described above, when the specialeffect image generating unit 11 c applies a special effect to one of asubject image and background image, and the synthetic image generatingunit 11 d synthesizes this special effect image to which this specialeffect is applied, with the other one of the subject image andbackground image, the display control unit 11 e controls the displayunit 7 to display this synthesized synthetic image, and controls thedisplay unit 7 to display information related to a special effect whichneeds to be applied to the subject image or background image, in orderfrom the highest advisability corresponding to the combination with aspecial effect applied immediately before, referring to the specialeffect information table T1 stored in the special effect informationstoring unit 9 d. Consequently, when a synthetic image is generated byapplying special effects individually to two items of image data, it ispossible to present, to the user, information which serves as a guide toselect a combination of special effects.

Further, with the first embodiment, the display control unit 11 econtrols the display unit 7 to first display special effect icons of acombination of the highest advisability, so that it is possible toobtain a synthetic image by applying special effects to the subjectimage and background image, respectively without missing a chance tocapture an image of the subject due to the user's operation of selectingthe special effects.

Second Embodiment

Next, a second embodiment will be described. With the above firstembodiment, the display control unit controls the display unit todisplay information related to a special effect which needs to beapplied to a subject image or background image in order from the highestadvisability corresponding to a combination with a special effectapplied immediately before, referring to the special effect informationtable. By contrast with this, with the second embodiment, a specialeffect information storing unit stores a second special effectinformation table which assigns an advisability corresponding to acombination of a shooting mode which can be set in the imaging apparatusand a special effect applied to a subject image or background image,and, when the shooting mode is set according to an input from the inputunit, the display control unit controls the display unit to displayinformation related to a special effect in order from the highestadvisability corresponding to a combination with a set shooting mode,referring to the second special effect information table. In addition,the imaging apparatus according to the second embodiment employs thesame configuration as the imaging apparatus 1 according to the abovefirst embodiment. Hence, with the second embodiment, a plurality ofshooting modes will be described which are provided in the imagingapparatus 1 and are set respectively according to various capturingscenes.

First, a plurality of shooting modes will be described which areprovided in the imaging apparatus 1 according to the second embodimentand which are set respectively according to various capturing scenes.The imaging apparatus 1 according to the second embodiment has shootingmodes of normal image capturing, panorama shooting, swing 3D shooting,3D shooting, HDR (High Dynamic Range) shooting and S strobe shooting. Inaddition, the above shooting modes are only examples, and are by nomeans limited to these.

FIG. 20 is a schematic diagram describing a capturing method of panoramashooting according to a shooting mode performed by the imaging apparatus1. As illustrated in FIG. 20, when the user moves the imaging apparatus1 from the left side to the right side, the imaging apparatus 1generates a plurality of items of image data continuously captured at aplurality of capturing positions. More specifically, as illustrated inFIG. 20, the imaging apparatus 1 generates images W31 to W33 ((a) ofFIG. 20). Then, the synthetic image generating unit 11 d generates onepanorama image W34 by overlaying and synthesizing ends of the images W31to W33 in the lateral direction ((b) of FIG. 20). By this means, theuser can obtain a panorama image W34 by capturing images while movingthe imaging apparatus 1. In addition, in (a) of FIG. 20, typical threeimages W31 to W33 of a plurality of images generated by the imaging unit2 are illustrated, and there are a plurality of images between theimages W31 and W33.

Further, the imaging apparatus 1 can perform 3D shooting and swing 3Dshooting in the shooting modes of the imaging apparatus 1 by capturingimages while being moved by the user. 3D shooting refers to continuouslycapturing a subject while the user moves the imaging apparatus 1, andgenerating two items of image data in which one ends of the respectivefield of views in left and right directions overlap. More specifically,as illustrated in FIG. 20, by continuously capturing a subject A1 whilebeing moved by the user from the left to the right, the imagingapparatus 1 generates two items of image data in which one ends of therespective field of views in the left and right directions overlap. Bythis means, the imaging apparatus 1 can generate three-dimensionalimages (hereinafter “3D image”) which the user virtually recognizes.Further, swing 3D shooting refers to generating a special effect 3Dimage by applying a special effect to a captured 3D image. Morespecifically, the image separating unit 11 b separates the subject imageand background image from the 3D image, and the special effect imagegenerating unit 11 c generates a special effect image by applyingspecial effects to the separated subject image and background imagerespectively. Then, the synthetic image generating unit 11 d synthesizesthe special effect images and generates a special effect 3D image. Bythis means, the imaging apparatus 1 can generate a special effect 3Dimage obtained by applying a special effect to a 3D image.

FIG. 21 is a schematic diagram describing a capturing method of HDRshooting according to a shooting mode performed by the imaging apparatus1. FIG. 22 is a view illustrating an example of an image generated bythe imaging apparatus 1 under the situation illustrated in FIG. 21. Inaddition, in FIG. 22, an image W40 is generated by the imaging apparatus1 under the situation illustrated in FIG. 21( a), and an image W41 isgenerated by the imaging apparatus 1 under the situation illustrated inFIG. 21( b). HDR shooting refers to synthesizing an image of an imagehaving brightness for a dark portion and an image having brightness fora bright portion and expanding a reproduction area of the brightness inorder to obtain an image of a large dynamic range.

As illustrated in FIG. 21, the imaging apparatus 1 first radiates lightof the light emitting unit 4 on a subject A3, captures the subject A3and generates the image W40 (FIG. 22( a)). Subsequently, the imagingapparatus 1 captures the subject A3 over an exposure time which is, forexample, doubled compared to normal capturing, and generates the imageW41 ((b) of FIG. 22).

Then, the image separating unit 11 b separates the subject images andbackground images from the image W40 and image W41, based on thedifference between the brightness value of the image W40 and brightnessvalue of the image W41. More specifically, the image separating unit 11b specifies and separates as the subject image an area having thedifference between the brightness value of the image W40 and brightnessvalue of the image W41 equal to or more than a threshold.

By contrast with this, the image separating unit 11 b specifies andseparates as the background image an area having the difference betweenthe brightness value of the image W40 and the brightness value of theimage W41 less than a threshold.

Subsequently, the synthetic image generating unit 11 d generates asynthetic image by synthesizing the subject image separated from theimage W40 and the background image separated from the image W41 by theimage separating unit 11 b.

By this means, the imaging apparatus 1 can generate a synthetic imagehaving optimal brightness value for the subject A3 and background.

Further, the imaging apparatus 1 can perform S strobe shooting ofcapturing an image by increasing strobe light emission compared tonormal capturing, and background blurring capturing of blurringprocessing the background by increasing an aperture value compared tonormal capturing. Meanwhile, S strobe refers to processing ofsynthesizing an image for which strobe light is emitted, and an imagefor which an exposure time is provided longer without emitting strobelight, preventing the background from becoming dark and reproducing thebackground and person correctly. Further, background blurring refers toprocessing of minimizing the change of the brightness of an image anddecreasing the contrast.

Processing performed by the imaging apparatus 1 according to the secondembodiment which has the above shooting modes will be described. Theimaging apparatus 1 according to the second embodiment performs the sameprocessing as in the above first embodiment (see FIG. 11) except menuoperation process and capturing process. Hence, menu operation processperformed by the imaging apparatus 1 according to the second embodimentwill be described, and then capturing process performed by the imagingapparatus 1 according to the second embodiment will be described.

FIG. 23 is a flowchart illustrating an outline of menu operation processperformed by the imaging apparatus 1 according to the second embodiment.

In FIG. 23, the display control unit 11 e first controls the displayunit 7 to display the menu icons (see FIG. 13) on the image displayed onthe display unit 7 (step S501), and the control unit 11 decides whetheror not the user operates the special shooting mode icon (step S502).When the user does not operate the special shooting mode icon within apredetermined time (step S502: No), the imaging apparatus 1 returns tothe main routine illustrated in FIG. 11. By contrast with this, when theuser operates the special shooting mode icon (step S502: Yes), theimaging apparatus 1 goes to step S503.

Subsequently, the display control unit 11 e controls the display unit 7to display submenu icons of the special shooting mode on the imagedisplayed on the display unit 7 (step S503). More specifically, asillustrated in FIG. 24, the display control unit 11 e controls thedisplay unit 7 to display, for example, a panorama icon Q21 whichreceives an input of a command signal for commanding the setting ofpanorama shooting in the imaging apparatus 1 and a swing 3D icon Q22which receives an input of a command signal for commanding the settingof swing 3D shooting in the imaging apparatus 1, on an image W50displayed on the display unit 7. In addition, although, in FIG. 24, thesubmenu icons of the shooting mode are only the panorama icon Q21 andswing 3D icon Q22, the submenu icons are by no means limited to these.More specifically, when the user operates the switch icon Q23 or Q24which receives an input of a command signal for commanding to switch theshooting mode, the display control unit 11 e controls the display unit 7to display another submenu icon, for example, a 3D shooting icon whichreceives an input of a command signal for commanding the setting of 3Dshooting in the imaging apparatus 1.

Subsequently, the control unit 11 decides whether or not the useroperates the swing 3D icon (step S504). When the user does not operatethe swing 3D icon within a predetermined time (step S504: No), theimaging apparatus 1 returns to the main routine illustrated in FIG. 11.By contrast with this, when the user operates the swing 3D icon (stepS504: Yes), the imaging apparatus 1 goes to step S505.

Then, the display control unit 11 e controls the display unit 7 todisplay an image selection icon and special effect icons on the imagedisplayed on the display unit 7 (step S505). More specifically, asillustrated in FIG. 25, the display control unit 11 e controls thedisplay unit 7 to display a subject selection icon Q26 as the imageselection icon, and to display a pop icon Q27 as information related toa special effect in order from the highest advisability corresponding tothe combination with the set shooting mode, referring to a secondspecial effect information table stored in the special effectinformation storing unit 9 d. In addition, in FIG. 25, the icon Q25indicates information related to the current shooting mode of theimaging apparatus 1 set by the user and information of an image to whicha special effect set by the user is applied.

Hereinafter, a method of displaying special effect icons which thedisplay control unit 11 e controls the display unit 7 to display will bedescribed with reference to FIG. 26. FIG. 26 is a view illustrating anexample of the second special effect information table stored by thespecial effect information storing unit 9 d.

As illustrated in FIG. 26, in the special effect information table T2,special effects are written which are applicable to shooting mode whichcan be set in the imaging apparatus 1. More specifically, the specialeffects which are applicable to the shooting mode include “miniature”,“pop”, “fish-eye”, “soft”, “sepia”, “black/white” and “sketch”. Further,numerical values (0 to 5) are written for advisabilities correspondingto combinations of shooting modes which can be set in the imagingapparatus 1 and special effects applied to a subject image or backgroundimage. More specifically, when the shooting mode is “swing 3D”, theadvisability is “5” for “pop” of the special effect. Further, when theshooting mode is “swing 3D”, the advisability is “0” for “fish-eye” ofthe special effect. The combination of this shooting mode and specialeffect having the advisability of “0” cannot provide a distinctivevisual effect of a special effect for the captured image.

Thus, with the second embodiment, the display control unit 11 e controlsthe display unit 7 to display special effect icons in order from thehighest advisability with the shooting mode, referring to the secondspecial effect information table T2 stored in the special effectinformation storing unit 9 d. Further, the display control unit 11 econtrols the display unit 7 to display only icons of special effectswhich can be used in combination with shooting modes. That is, thedisplay control unit 11 e controls the display unit 7 not to displayspecial effect icons having the advisability “0” of the shooting modeand special effect.

Referring back to FIG. 23, in step S506, the control unit 11 decideswhether or not the user operates the end icon Q29 (see FIG. 25). Whenthe user operates the end icon Q29 (step S506: Yes), the imagingapparatus 1 returns to the main routine illustrated in FIG. 11. Bycontrast with this, when the user does not operate the end icon Q29(step S506: No), the imaging apparatus 1 goes to step S507.

Subsequently, the control unit 11 decides whether or not the useroperates the image selection icon (step S507). More specifically, asillustrated in FIG. 25, the control unit 11 decides whether or not thesubject selection icon Q26 which receives an input of a command signalfor switching to the setting of a special effect applied to the subjectimage is operated. When the user does not operate the image selectionicon (step S507: No), the control unit 11 decides whether or not theuser operates the special effect icon (step S508). More specifically, asillustrated in FIG. 25, the control unit 11 decides whether or not theuser operates the pop icon Q27. When the user operates the specialeffect icon (step S508: Yes), the control unit 11 sets the specialeffect for the background image according to the special effect iconoperated by the user and stores this set setting content in thetemporary image data storing unit 9 c (step S509), and the imagingapparatus 1 returns to step S505.

A case (step S508: No) will be described where, in step S508, the userdoes not operate the special effect icon. In this case, the control unit11 decides whether or not the user performs an operation to switch aspecial effect icon (step S510). More specifically, as illustrated inFIG. 25, the control unit 11 decides whether or not the user operatesthe switch icon Q28 which receives an input of a command signal forcommanding to switch to a special effect icon of the second highestadvisability from the special effect icon currently displayed on thedisplay unit 7. When the user does not perform an operation to switch aspecial effect icon within a predetermined time (step S510: No), theimaging apparatus 1 returns to step S505. By contrast with this, whenthe user performs an operation to switch a special effect icon (stepS510: Yes), the display control unit 11 e controls the display unit 7 toswitch a special effect icon of the second highest advisability such asa special effect icon of “black/white” to display, referring to thespecial effect information table T2 stored in the special effectinformation storing unit 9 d (step S511), and the imaging apparatus 1returns to step S508.

A case (step S507: Yes) will be described where, in step S507, the useroperates the image selection icon. In this case, the display controlunit 11 e controls the display unit 7 to display the image selectionicon and special effect icons on the image displayed on the display unit7 referring to the special effect information table stored in thespecial effect information storing unit 9 d (step S512). Morespecifically, as illustrated in FIG. 27, the display control unit 11 econtrols the display unit 7 to display an image selection icon Q30, anddisplay a normal icon Q31, a sepia icon Q32 and a black/white icon Q33referring to the special effect information table stored in the specialeffect information storing unit 9 d. In addition, in FIG. 27, the imageselection icon Q30 is added information of the current shooting mode ofthe imaging apparatus 1 set by the user and a special effect such as ablack/white effect set in the background, and is displayed. By thismeans, the user can set a special effect applied to the subject imagewhile checking the special effect set for the background image.

Hereinafter, a method of displaying special effect icons which thedisplay control unit 11 e controls the display unit 7 to display will bedescribed with reference to FIG. 28. FIG. 28 is a view illustrating anexample of a special effect information table stored by the specialeffect information storing unit 9 d.

As illustrated in FIG. 28, in the special effect information table T3,special effects are written which are applicable to a subject image andbackground image, respectively. More specifically, in the special effectinformation table T3, numerical values (0 to 5) are written foradvisabilities of combinations of special effects to generate asynthetic image by applying special effects to a subject image andbackground image, respectively. More specifically, when the specialeffect for the subject image is “normal”, the advisability is “5” for“normal”, “soft” and “black/white” of special effects for the backgroundimage. Further, the advisability is “4” for “sepia” and the advisabilityis “0” for “pop”, “fish-eye”, “sketch” and “miniature”. The combinationof special effects having the advisability of “0” cannot provide adistinct visual effect of a special effect for the captured image.Hence, with the second embodiment, the display control unit 11 econtrols the display unit 7 to display only icons of the special effectswhich can be used in combination, referring to the special effectinformation table T3 stored in the special effect information storingunit 9 d. That is, the display control unit 11 e controls (limits) thedisplay unit 7 not to display special effect icons having theadvisability of “0” for a combination of special effects. By this means,when special effects are set to the background image and subject image,respectively to generate a synthetic image, the user can reliablyprevent an image from becoming unnatural.

Referring back to FIG. 23, in step S513, the control unit 11 decideswhether or not the user operates the special effect icon. When the useroperates a special effect icon (step S513: Yes), the control unit 11sets a special effect of a subject image corresponding to the specialeffect icon operated by the user, and stores this set setting content inthe temporary image data storing unit 9 c (step S514), and the imagingapparatus 1 returns to step S505.

By contrast with this, when the user does not operate the special effecticon (step S513: No), the control unit 11 decides whether or not theuser performs an operation to switch a special effect icon (step S515),and, more specifically, as illustrated in FIG. 27, the control unit 11decides whether or not the user touches the switch icon Q34 whichreceives an input of a command signal for commanding to switch to aspecial effect icon of the second highest advisability from a specialeffect icon currently displayed on the display unit 7. When the userdoes not perform an operation to switch the special effect icon within apredetermined time (step S515: No), the imaging apparatus 1 returns tostep S505. By contrast with this, when the user performs an operation toswitch the special effect icon (step S515: Yes), the display controlunit 11 e controls the display unit 7 to switch the special effect iconof the second highest advisability such as the special effect icon of“pop” to display, referring to the special effect information table T3stored in the special effect information storing unit 9 d (step S516),and the imaging apparatus 1 returns to step S513.

Next, capturing process performed by the imaging apparatus 1 accordingto the second embodiment will be described. FIG. 29 is a flowchartillustrating an outline of capturing process. In addition, a case willbe described below where the shooting mode of the imaging apparatus 1 isset to swing 3D shooting.

As illustrated in FIG. 29, the control unit 11 first decides whether ornot the shooting mode of the imaging apparatus 1 is set to swing 3Dshooting (step S601). When the shooting mode of the imaging apparatus 1is set to swing 3D shooting (step S601: Yes), the control unit 11decides whether or not the user starts moving the imaging apparatus 1(step S602). When the imaging apparatus 1 does not start moving (stepS602: No), the control unit 11 repeats this decision. By contrast withthis, when the user starts moving the imaging apparatus 1 (step S602:Yes), the imaging apparatus 1 goes to step S603.

Subsequently, the imaging apparatus 1 continuously captures images atvery short time intervals (step S603), and the control unit 11 controlsthe temporary image data storing unit 9 c to temporarily store thecaptured images (step S604).

Then, the control unit 11 decides whether or not the imaging apparatus 1stops moving (step S605). When the imaging apparatus 1 does not stopmoving (step S605: No), the imaging apparatus 1 returns to step S603. Bycontrast with this, when the imaging apparatus 1 stops moving (stepS605: Yes), the imaging apparatus 1 goes to step S606.

Subsequently, the control unit 11 decides whether or not the imageseparating unit 11 b can separate a subject image and background imagefrom a plurality of captured images (step S606). When the imageseparating unit 11 b cannot separate the subject image and backgroundimage from a plurality of captured images (step S606: No), the displaycontrol unit 11 e controls the display unit 7 to display a warning thatthe subject image and background image cannot be separated from thecaptured images (step S607), and the imaging apparatus 1 returns to themain routine illustrated in FIG. 11.

A case (step S606: Yes) will be described where, in step S606, the imageseparating unit 11 b can separate a subject image and background imagefrom the captured images. In this case, the image separating unit 11 bseparates the subject image and background image from the capturedimages (step S608), and the special effect image generating unit 11 cgenerates a special effect image by applying set special effects to thesubject image and background image stored in the temporary image datastoring unit 9 c (step S609).

Subsequently, the synthetic image generating unit 11 d generates asynthetic image by synthesizing special effect images of the respectivesubject image and background image generated by the special effect imagegenerating unit 11 c (step S610), and the display control unit 11 econtrols the display unit 7 to display a 3D special effect image usingthe synthetic image generated by the synthetic image generating unit 11d (step S611). More specifically, as illustrated in FIG. 30, the displaycontrol unit 11 e controls the display unit 7 to display a 3D imageusing two items of image data of a synthetic image WL2 and syntheticimage WR2. By this means, the user can virtually see the 3D imageobtained by applying special effects to the subject image and backgroundimage.

Then, the control unit 11 controls the image data storing unit 9 a tostore the image currently displayed on the display unit 7 (step S612),and the imaging apparatus 1 returns to the main routine illustrated inFIG. 11.

A case (step S601: No) will be described where, in step S601, theshooting mode of the imaging apparatus 1 is not set to swing 3Dshooting. In this case, the imaging apparatus 1 captures the subject(step S613), and the display control unit 11 e controls the display unit7 to REC-view display the captured image (step S614).

Then, the control unit 11 controls the image data storing unit 9 a tostore the captured image (step S615), and the imaging apparatus 1returns to the main routine illustrated in FIG. 11.

According to the above described second embodiment, the special effectinformation storing unit 9 d stores the second special effectinformation table T2 which assigns the advisability corresponding to acombination of the shooting mode which can be set in the imagingapparatus 1 and special effect applied to the subject image orbackground image, and, when the shooting mode is set according to aninput from the touch panel 8, the display control unit 11 e controls thedisplay unit 7 to display information related to a special effect inorder from the highest advisability corresponding to the combinationwith the set shooting mode, referring to the second special effectinformation table T2. By this means, when the user performs capturingusing the shooting mode and special effect in combination, it ispossible to present, to the user, information which serves a guide toselect the combination of the shooting mode and special effect.

Further, according to the second embodiment, the display control unit 11e controls the display unit 7 to display only special effect iconscorresponding to special effects which can be used in combination withshooting modes. By this means, when capturing an image using a shootingmode and special effect in combination, the user can reliably preventimages to capture from becoming unnatural.

In addition, although, with the above second embodiment, the user movesthe imaging apparatus 1 to generate two items of image data in which oneends of respective field of views in the left and right directionsoverlap, for example, it is possible to generate two items of image datain which one ends of respective field of views in the left and rightdirections overlap by forming an imaging unit with two optical systems.

First Modification of First and Second Embodiments

Further, with the first and second embodiments, the display control unit11 e may control the display unit 7 to emphasize and display a specialeffect icon in order from the highest advisability of a combination witha special effect according to a display mode of a changed color and/orsize, referring to the special effect information table stored in thespecial effect information storing unit 9 d. By this means, the user canintuitively select an optimal combination of special effects basedinformation which serves as a guide to select a combination of specialeffects.

Second Modification of First and Second Embodiments

Further, although, with the first and second embodiments, the syntheticimage generating unit 11 d synthesizes special effect images obtained byrespectively applying special effects to a subject image and backgroundimage separated by the image separating unit 11 b, for example, thesynthetic image generating unit 11 d may generate a synthetic image bygenerating two special effect images by applying special effects to, forexample, an image (for example, compressed data, uncompressed data, orcompressed data or uncompressed data classified per aspect ratio) storedin the image data storing unit 9 a and an image immediately after animage is captured by multiple exposures and synthesizing these twogenerated special effect images.

Third Modification of First and Second Embodiments

Further, although, with the first and second embodiments, the posturedetecting unit 3 detects the posture state of the imaging apparatus 1,the posture detecting unit 3 may receive an operation signal of a touchoperation of switching various shooting modes or various settings of theimaging apparatus 1 by detecting an acceleration produced when the usertaps the display screen of the display unit 7, and output this operationsignal to the control unit 11.

Fourth Modification of First and Second Embodiments

Further, with the first and second embodiments, it is possible to updateor rewrite the special effect information table stored in the specialeffect information storing unit 9 d by the connecting communication unit10 to an external processing device such as a personal computer orserver. By this means, the display control unit 11 e can display specialeffect icons according to the advisability which takes into account acombination of newly added special effects and shooting modes. Further,by storing advisabilities of combinations of newly added special effectsand shooting modes in a storage medium such as a memory card which isattachable to the storing unit 9, the control unit 11 may acquire theadvisability from the storage medium and store the advisability in thespecial effect information storing unit 9 d to update or rewrite aspecial effect information table.

Third Embodiment

Next, a third embodiment will be described. An imaging apparatusaccording to the third embodiment differs from the above embodiments inconfigurations of the control unit and storing unit. Further, theimaging apparatus according to the third embodiment differs from theabove embodiments in the operation of the imaging apparatus. Hence,hereinafter, the configurations of the storing unit and control unit ofthe imaging apparatus according to the third embodiment will bedescribed, and then the operation of the imaging apparatus according tothe third embodiment will be described. In addition, the same componentswill be assigned the same reference numerals in the drawings.

FIG. 31 is a block diagram illustrating a configuration of the imagingapparatus according to the third embodiment. As illustrated in FIG. 31,an imaging apparatus 100 has the imaging unit 2, the posture detectingunit 3, the light emitting unit 4, the clock 5, the operation input unit6, the display unit 7, the touch panel 8, the communication unit 10, astoring unit 109 and a control unit 110.

The storing unit 109 is provided inside the imaging apparatus 1 in afixed manner, and is formed using, for example, a semiconductor memorysuch as a flash memory or RAM. The storing unit 109 has: an image datastoring unit 109 a which stores image data continuously captured by theimaging unit 2; a program storing unit 109 b which stores variousprograms executed by the imaging apparatus 1 and capturing programsaccording to one embodiment; a shooting mode information storing unit109 c which stores information related to shooting modes for which framerates of capturing suitable for each capturing scene are set; and anadvisability information storing unit 109 d which stores an advisabilityinformation table which assigns an advisability corresponding to acombination of special effects and a frame rate of the shooting modewhen a special effect is applied to an image. In addition, the storingunit 109 may have a function of a storage medium interface which readsinformation stored in the storage medium while storing the informationin a storage medium such as a memory card attached from an outside.

The control unit 110 is formed with, for example, a CPU. The controlunit 110 reads a program from the program storing unit 109 b of thestoring unit 109 according to, for example, an operation signal from theoperation input unit 6 to execute, and performs overall control of theoperation of the imaging apparatus 100 by, for example, commanding eachunit forming the imaging apparatus 100 or transferring data. The controlunit 110 has an image processing unit 110 a, a shooting mode settingunit 110 b, a special effect image generating unit 110 c and a displaycontrol unit 110 d.

The image processing unit 110 a applies various image processings toimage data outputted from the signal processing unit 2 h to output tothe storing unit 109. More specifically, the image processing unit 110 aapplies processings such as edge enhancement, color correction and γcorrection to image data outputted from the signal processing unit 2 h.

The shooting mode setting unit 110 b sets one of a plurality of shootingmodes for which a frame rate of capturing suitable for each capturingscene is set in advance. More specifically, the shooting mode settingunit 110 b sets the shooting mode of the imaging apparatus 100 from aplurality of shooting modes stored in the shooting mode informationstoring unit 109 c based on an operation signal inputted from, forexample, the mode switch 6 c or the menu switch 6 e.

The special effect image generating unit 110 c generates a specialeffect image by applying special effect corresponding to informationreceived by the touch panel 8 as input, to an image corresponding toimage data generated by the imaging unit 2. More specifically, thespecial effect image generating unit 110 c generates a special effectimage by applying to image data continuously generated by the imagingunit 2 a special effect obtained by using various optical filters suchas a color conversion filter or contrast adjustment filter in the frontsurface of the lens unit 2 a of the imaging unit 2, or the same effectas a special obtained by using a shooting lens for a specific use suchas a fish-eye lens or a tilt-shift mechanism attached shooting lens. Forexample, the special effect image generating unit 110 c applies, toimage data, processing including one of a plurality of imageprocessings, tone curve process, blurring process, shading addingprocess, image synthesizing process, noise superimposing process, andsaturation adjusting process. In addition, the special effect imagegenerating unit 110 c may generate a special effect image by applyingtwo or more special effects of different effects to image data generatedby the imaging unit 2. Further, the special effect image generating unit110 c may generate a special effect image by applying a special effectcorresponding to an operation signal inputted from the special effectinput switch 6 d, to an image corresponding to image data generated bythe imaging unit 2.

The display control unit 110 d controls the display unit 7 to performdisplay. The display control unit 110 d controls the display unit 7 todisplay in time sequences the images sequentially generated at a framerate of the shooting mode set by the shooting mode setting unit 110 b,and controls the display unit 7 to display information related to aspecial effect which the special effect image generating unit 110 cneeds to apply to the images, in order from the highest advisabilitycorresponding to a combination with the frame rate of the shooting mode,referring to the advisability information table stored in theadvisability information storing unit 109 d. Further, the displaycontrol unit 110 d controls the display unit 7 to display informationrelated to special effects by means of icons. More specifically, thedisplay control unit 110 d controls the display unit 7 to displayspecial effect icons corresponding to special effects in order from thehighest advisability corresponding to the combination with the framerate of the shooting mode, referring to the advisability informationtable.

Processing performed by the imaging apparatus 100 employing the aboveconfiguration will be described. FIG. 32 is a flowchart illustrating anoutline of processing performed by the imaging apparatus 100. Inaddition, video capturing will be described below as an example.

In FIG. 32, the control unit 110 first decides whether or not the powersource of the imaging apparatus 100 is turned on (step S1101). When thepower source of the imaging apparatus 100 is not turned on (step S1101:No), the imaging apparatus 100 finishes the processing. By contrast withthis, when the power source of the imaging apparatus 100 is turned on(step S1101: Yes), the imaging apparatus 100 goes to step S1102.

Subsequently, the control unit 110 decides whether or not the imagingapparatus 100 is set to a shooting mode (step S1102). When the imagingapparatus 100 is set to the shooting mode (step S1102: Yes), the controlunit 110 decides whether or not a release signal for commanding videocapturing is inputted from the release switch 6 b (step S1103). Whenreceiving the release signal for commanding video capturing as input(step S1103: Yes), the imaging apparatus 100 starts video capturingaccording to the set shooting mode and special effect (step S1104). Thisvideo capturing is directed to storing, in the image data storing unit109 a, special effect image data obtained by applying a special effectfrom the special effect image generating unit 110 c to image datacontinuously generated by the imaging unit 2 at a frame rate of the setshooting mode in menu operation process (described below) or image datacontinuously generated by the imaging unit 2. By contrast with this,when receiving no release signal for commanding video capturing as input(step S1103: No), the imaging apparatus 100 goes to step S1105.

Then, the display control unit 110 d controls the display unit 7 todisplay live view images of an image corresponding to image datacontinuously generated by the imaging unit 2 (step S1105).

FIG. 33 is a view illustrating an example of a situation when theimaging apparatus 100 captures an image of a subject, and an imagedisplayed on the display unit 7. FIG. 33 illustrates one typical imageW_(n) (n=natural number) of live view images sequentially displayed onthe display unit 7 in time sequences. As illustrated in FIG. 33, thecamera man captures a subject O1 while checking a live view image suchas the image W_(n) displayed on the display unit 7.

In step S1106, when the menu switch 6 e is operated, the control unit110 decides whether or not a command signal for commanding to display anoperation menu is inputted from the menu switch 6 e. When receiving acommand signal for commanding to display an operation menu as input fromthe menu switch 6 e (step S1106: Yes), the imaging apparatus 100executes menu operation process for setting the shooting mode andspecial effect of the imaging apparatus 100 (step S1107), and goes tostep S1108. By contrast with this, when receiving no command signal forcommanding to display an operation menu from the menu switch 6 e asinput (step S1106: No), the imaging apparatus 100 goes to step S1108.

Subsequently, the control unit 110 decides whether or not the imagingapparatus 100 is capturing an image (step S1108). When the imagingapparatus 100 is not capturing an image (step S1108: No), the imagingapparatus 100 returns to step S1101. By contrast with this, when theimaging apparatus 100 is capturing an image (step S1108: Yes), thecontrol unit 110 decides whether or not an operation to finish capturingis performed (step S1109). More specifically, when the release switch 6b is operated, the control unit 110 decides whether or not a commandsignal for commanding to finish capturing is inputted from the releasesignal 6 b. When an operation to finish capturing is not performed (stepS1109: No), the imaging apparatus 100 returns to step S1105. By contrastwith this, when an operation to finish capturing is performed (stepS1109: Yes), the imaging apparatus 100 goes to step S1110.

Then, the control unit 110 finishes video capturing of the imagingapparatus 100 (step S1110), and creates an image file of captured imagedata (step S1111). This image file stores, for example, the capturedimage data, captured shooting mode information, frame rate informationof the captured video, special effect presence/absence information,special effect information, special effect count information and dateinformation.

Subsequently, the control unit 110 initializes the set shooting mode andspecial effect (step S1112), and the imaging apparatus 100 returns tostep S1101. More specifically, the control unit 110 sets varioussettings of the imaging apparatus 100 set in menu operation process, tothe initial state set upon shipping from a manufacturer.

A case (step S1102: No) will be described where, in step S1102, theshooting mode is not set in the imaging apparatus 100. In this case, theimaging apparatus 100 executes playback/display process of playing thecaptured image data (step S1113), and returns to step S1101.

Next, the menu operation process illustrated in FIG. 32 will bedescribed. FIG. 34 is a flowchart illustrating an outline of menuoperation process in step S1107 illustrated in FIG. 32.

As illustrated in FIG. 34, the display control unit 110 d first controlsthe display unit 7 to display an imaging menu (step S1201). Morespecifically, as illustrated in FIG. 35, the display control unit 110 dcontrols the display unit 7 to display a shooting mode menu icon Q101and a special effect menu icon Q102 as an operation menu M1. Theshooting mode menu icon Q101 receives an input of a command signal forcontrolling the display unit 7 to display information related to aplurality of shooting modes for which a frame rate of capturing suitablefor each capturing scene is set in advance. The special effect menu iconQ102 receives an input of a command signal for controlling the displayunit 7 to display information related to a plurality of special effectswhich are applicable to image data generated by the imaging unit 2. Inaddition, the display control unit 110 d may control the display unit 7to display an icon corresponding to information related to, for example,a strobe light menu or exposure setting menu as part of the operationmenu M1.

After the display control unit 110 d controls the display unit 7 todisplay the operation menu M1, the control unit 110 decides whether ornot the shooting mode menu icon Q101 is operated (step S1202). Morespecifically, as illustrated in FIG. 35, when the user touches theshooting mode menu icon Q101 displayed on the image W_(n) on the displayunit 7, the control unit 110 decides whether or not the shooting modemenu Q101 is operated according to the signal received by the touchpanel 8 as input. When the shooting mode menu icon Q101 is not operated(step S1202: No), the imaging apparatus 100 goes to step S1207(described below). By contrast with this, when the shooting mode menuicon Q101 is operated (step S1202: Yes), the display control unit 110 dcontrols the display unit 7 to display a list of shooting modes (stepS1203), and the imaging apparatus 100 goes to step S1204.

In step S1204, the control unit 110 decides whether or not a shootingmode is selected following the change of the frame rate. Morespecifically, when the user operates the special effect input switch 6d, the control unit 110 decides whether or not the shooting mode isselected from the list of shooting modes displayed on the display unit 7following the change of the frame rate. When the shooting mode isselected following the change of the frame rate (step S1204: Yes), theshooting mode setting unit 110 b sets the shooting mode selectedfollowing the change of the selected frame rate, in the imagingapparatus 100 (step S1205), and the imaging apparatus 100 goes to stepS1207. By contrast with this, when the shooting mode is not selectedfollowing the change of the frame rate after a predetermined time, forexample, 30 seconds passes (step S1204: No), the shooting mode settingunit 110 b sets a normal shooting mode in the imaging apparatus 100(step S1206), and the imaging apparatus 100 goes to step S1207.Meanwhile, the normal shooting mode refers to a shooting mode ofcapturing an image at a frame rate of 30 fps.

Subsequently, the control unit 110 decides whether or not the specialeffect menu icon Q102 is operated (step S1207). When the special effectmenu icon Q102 is operated (step S1207: Yes), the control unit 110decides whether or not the shooting mode selected following the changeof the frame rate is set in the imaging apparatus 100 (step S1208). Whenthe shooting mode selected following the change of the frame rate is setin the imaging apparatus 100 (step S1208: Yes), the display control unit110 d controls the display unit 7 to display information related to aspecial effect which the special effect image generating unit 110 cneeds to apply to an image displayed on the display unit 7, in orderfrom the highest advisability corresponding to a combination with theframe rate of the shooting mode set by the shooting mode setting unit110 b, referring to the advisability table stored in the advisabilityinformation storing unit 109 d (step S1209). More specifically, asillustrated in FIG. 36, the display control unit 110 d controls thedisplay unit 7 to display a miniature icon Q111 and a pop icon Q112.

Hereinafter, a method of displaying special effect icons which thedisplay control unit 110 d controls the display unit 7 to display willbe described with reference to FIG. 37. FIG. 37 is a view illustratingan example of an advisability information table stored by theadvisability information storing unit 109 d.

As illustrated in FIG. 37, in the advisability table T11, (0 to 2) arewritten for advisabilities which correspond to combinations of specialeffects which the special effect image generating unit 110 c can applyto image data and frame rates of shooting modes. More specifically, whenthe shooting mode is “fast forward 2 (4 fps) shooting mode”, theadvisability is “2” for a combination with a special effect of“miniature”, “pop”, “sepia” or “black/white”. Further, the advisabilityis “1” for a combination with “fish-eye”, “soft” or “sketch”. That is,in the advisability information table T11, a higher numerical value isset for a better combination of a frame rate of a shooting mode and aspecial effect.

Hereinafter, each special effect and the advisability for a combinationof a frame rate of a shooting mode and a special effect illustrated inFIG. 37 will be described.

First, each special effect illustrated in FIG. 37 will be described.

Miniature provides an effect of adjusting focus on the center of thescreen and decreasing and blurring processing the contrast in thesurrounding such as upper and lower parts. More specifically, withminiature, shading adding process and image synthesizing process areperformed.

Pop provides an effect of emphasizing a color impressively andcolorfully. For example, processing of improving the hue and saturationof image data is performed.

Fish-eye provides an effect of finishing an image by distorting a linein an arc shape as if the line is captured using a fish-eye lens. Forexample, processing of changing image data to an image distorted by, forexample, coordinate conversion is performed.

Soft provides an effect of finishing the entire image softly as if theimage was captured using a soft focus lens.

Sepia provides an effect of finishing a sepia and monochrome image. Forexample, processing of decreasing levels of the white balance and tonecurve is performed.

Black/white provides an effect of finishing a black and white imagerepresented by a gray tone. For example, processing of decreasing thelevel of the white balance is performed.

Sketch provides an effect of finishing a sketchy image in which theprofile is drawn by line drawing.

Next, the advisability of a combination of a frame rate of a shootingmode and a special effect will be described. Miniature provides aneffect when an actual scenery needs to be finished like a toy world, andthe advisability is “2” for the fast-forward 2 shooting mode (4 fps) andfast-forward 3 shooting mode (8 fps). Furthermore, miniature makes avideo unnatural upon playback when a frame rate is too high, andtherefore the advisability is “1” for a fast-forward 1 shooting mode (2fps). Furthermore, miniature is a special effect suitable for capturinga still image, and therefore the advisability is “1” for the normalshooting mode “30 fps”. Further, miniature takes time to performprocessing applied to an image, and therefore the advisability is “0”for a 3D shooting mode (60 fps), a slow motion 1 shooting mode (120 fps)and a slow motion 2 shooting mode (240 fps).

Pop provides an effect when an image is emphasized colorfully andimpressively and finished smoothly, and therefore the advisability is“2” for the fast-forward 2 shooting mode, fast-forward 3 shooting mode,normal shooting mode and 3D shooting mode. Further, pop has difficultyin emphasizing an effect at a slow rate, and therefore the advisabilityis “1” for the slow motion 1 shooting mode and slow motion 2 shootingmode. Furthermore, pop makes a video unnatural upon playback when thetempo is too fast, and therefore the advisability is “1” for thefast-forward 1 shooting mode.

Fish-eye makes it possible to express that a scenery gradually changeswhen the scenery is captured while walking, and therefore theadvisability is “2” for the fast-forward 1 shooting mode. In addition,fish-eye is a special effect suitable for capturing a still image, andtherefore the advisability is “1” for the fast-forward 2 shooting mode,fast-forward 3 shooting mode and normal shooting mode. Furthermore,fish-eye takes time to perform processing applied to an image, andtherefore the advisability is “0” for the 3D shooting mode, slow motion1 shooting mode and slow motion 2 shooting mode.

Soft makes it possible to express that an entire image is finishedsoftly and a flashback is more emphasized, and therefore theadvisability is “2” for the slow motion 1 shooting mode, slow motion 2shooting mode and normal shooting mode. Further, soft has difficulty inexpressing flashbacks, and therefore the advisability is “1” for thefast-forward 1 shooting mode, fast-forward 2 shooting mode andfast-forward 3 shooting mode. Furthermore, soft has difficulty in makingthe user recognize a three-dimensional image because the informationamount which the left and right eyes can recognize is a little, andtherefore the advisability is “1” for the 3D shooting mode.

Sepia and black/white can provide an expression such as an old video,and therefore the advisability is “2” for the fast-forward 2 shootingmode and fast-forward 3 shooting mode. Further, sepia and black/whitemake it possible to emphasize and display emphasized flashbacks, andtherefore the advisability is “2” for the normal shooting mode, slowmotion 1 shooting mode and slow motion 2 shooting mode. Furthermore,sepia and black/white have difficulty in making the user recognize athree-dimensional image because the information amount which the leftand right eyes can recognize is a little, and therefore the advisabilityis “1” for the 3D shooting mode. Furthermore, sepia and black/white makea video unnatural upon play, and therefore the advisability is “1” forthe fast-forward 1 shooting mode.

Sketch makes it possible to provide an expression like an animation bydrawing a profile of a subject by line drawing, and therefore theadvisability is “2” for the fast-forward 3 shooting mode. Further,sketch is a special effect suitable to capture a still image, andtherefore the advisability is “1” for the fast-forward 1 shooting mode,fast-forward 2 shooting mode, normal shooting mode, slow motion 1shooting mode and slow motion 2 shooting mode. Furthermore, sketch hasdifficulty in making the user recognize a three-dimensional imagebecause the information amount which the left and right eyes canrecognize is a little, and therefore the advisability is “0” for the 3Dshooting mode.

Thus, with the advisabilities in the advisability information table T11,a higher numerical value is set stepwise for a better combination of aframe rate of a shooting mode and a special effect. Further, theadvisability in the advisability information table T11 is set to “0” forunsuitable combinations of frame rates of shooting modes and specialeffects. By this means, when the display control unit 110 d controls thedisplay unit 7 to display information related to special effects asspecial effect icons, the display control unit 110 d controls thedisplay unit 7 to display the information in order from the highestadvisability of the combination with the frame rate of the shooting modeset by the shooting mode setting unit 110 b, referring to theadvisability information table T11 stored in the advisabilityinformation storing unit 109 d.

Further, the display control unit 110 d controls the display unit 7 todisplay only information related to a special effect which can be usedin combination with the frame rate of the shooting mode set by theshooting mode setting unit 110 b. For example, when the shooting modesetting unit 110 b sets the slow motion 1 shooting mode, the displaycontrol unit 110 d controls the display unit 7 to display only thespecial effect icons related to pop, soft, sepia, black/white andsketch. As a result, the imaging apparatus 100 can present, to the user,information which serves as a guide for an optimal combination ofspecial effects for image data generated at the set frame rate of theshooting mode. In addition, the numerical values illustrated in FIG. 37are only examples and are by no means limited to these, and a numericalvalue of the advisability can be changed by, for example, the specialeffect input switch 6 d. Further, a lower advisability may be setstepwise for a better combination of a special effect and frame rate ofthe shooting mode.

Referring back to FIG. 34, in step S1210, the control unit 110 decideswhether or not an operation to switch a special effect icon isperformed. More specifically, as illustrated in FIG. 36, the controlunit 110 decides whether the user touches a switch icon Q113 or a switchicon Q114 which receives an input of a command signal for commanding toswitch to a special effect icon of the second highest advisability fromthe special effect icon currently displayed on the display unit 7. Whenthe user performs the operation to switch the special effect icon (stepS1210: Yes), the display control unit 110 d controls the display unit 7to switch special effect icons related to the special effect icons ofthe second highest advisability to display, referring to theadvisability information table T11 (step S1211). Then, the imagingapparatus 100 returns to step S1210. More specifically, as illustratedin FIG. 38, the display control unit 110 d controls display unit 7 todisplay the miniature icon Q111 and pop icon Q112 to the sepia icon Q121and black/white icon Q122 ((a)→(b) of FIG. 38). By this means, the usercan select a special effect in order from the highest advisability byswitching the special effect.

A case (step S1210: No) will be described where, in step S1210, theoperation to switch the special effect icon is not performed. In thiscase, the control unit 110 decides whether or not the special effecticon is operated (step S1212). When the special effect icon is operated(step S1212: Yes), the special effect image generating unit 110 cgenerates a special effect image by applying a special effect such aspop corresponding to the special effect icon which the user touches, toimage data generated by the imaging unit 2 (step S1213).

Subsequently, the display control unit 110 d controls the display unit 7to display a special effect image generated by the special effect imagegenerating unit 110 c (step S1214). More specifically, as illustrated inFIG. 39, the display control unit 110 d controls the display unit 7 todisplay a special effect image W_(m) (m=integer) by applying a specialeffect such as pop, to image data by the special effect image generatingunit 110 c. By this means, live view images displayed on the displayunit 7 are applied special effects, so that the user can capture imagesin real time while checking a special effect and composition forcapturing. In addition, although, in FIG. 39, an image is expressed bybold lines to show a special effect of pop, this image is only anexample. After step S1214, the imaging apparatus 100 returns to the mainroutine illustrated in FIG. 32.

A case (step S1212: No) will be described where, in step S1212, aspecial effect icon is not operated within a predetermined time, forexample, within 30 seconds. In this case, the display control unit 110 dcontrols the display unit 7 to finish displaying the special effecticons displayed on the display unit 7 (step S1215), and the imagingapparatus 100 returns to the main routine illustrated in FIG. 32.

A case (step S1208: No) will be described where, in step S1208, ashooting mode selected following a change of a frame rate is not set inthe imaging apparatus 100. In this case, the display control unit 110 dcontrols the display unit 7 to display a list of special effect iconsrelated to special effects (step S1216), and the imaging apparatus 100goes to step S1212.

A case (step S1207: No) will be described where, in step S1207, aspecial effect menu icon is not operated. In this case, the control unit110 decides whether or not a predetermined time, for example, 30 secondspasses after the display unit 7 displays the operation menu M1 (stepS1217). When a predetermined time does not pass after the display unit 7displays the operation menu M1 (step S1217: No), the imaging apparatus100 returns to step S1201. By contrast with this, when a predeterminedtime passes after the display unit 7 displays the operation menu M1(step S1217: Yes), the display control unit 110 d controls the displayunit 7 to finish displaying the operation menu M1 displayed on thedisplay unit 7 (step S1218), and the imaging apparatus 100 returns tothe main routine illustrated in FIG. 32.

Next, playback/display process in step S1113 illustrated in FIG. 32 willbe described. FIG. 40 is a flowchart illustrating an outline ofplayback/display process illustrated in FIG. 32.

As illustrated in FIG. 40, the display control unit 110 d first controlsthe display unit 7 to display an image selection screen collectivelydisplaying a plurality of items of image data stored in the image datastoring unit 109 a (step S1301).

Subsequently, the control unit 110 decides whether or not the usertouches the touch panel 8 to select an image from the image selectionscreen displayed on the display unit 7 (step S1302). When the user doesnot select an image from the image selection screen after apredetermined time, for example, 30 seconds passes (step S1302: No), theimaging apparatus 100 goes to step S1306 (described below). By contrastwith this, when the user selects an image from the image selectionscreen (step S1302: Yes), the control unit 110 decides whether or notthe selected image is a video (step S1303). More specifically, thecontrol unit 110 decides whether or not the image is a video, based onvideo information, for example, a recording mode, stored in an imagefile of the selected image. When the selected image is a video (stepS1303: Yes), the imaging apparatus 100 goes to step S1304 (describedbelow). By contrast with this, when the selected image is not a video(step S1303: No), the imaging apparatus 100 goes to step S1307(described below).

A case (step S1303: Yes) will be described where, in step S1303, theselected image is a video. In this case, the display control unit 110 dplays a video based on the frame rate of the shooting mode stored in theimage file of the video (step S1304), and the control unit 110 decideswhether or not playback of the video is finished (step S1305). Whenplayback of the video is not finished (step S1305: Yes), the controlunit 110 repeats this decision. By contrast with this, when playback ofthe video is finished (step S1305: Yes), the imaging apparatus 100 goesto step S1306.

Subsequently, the control unit 110 decides whether or not an operationto finish playing the image is performed (step S1306). When theoperation to finish playing the image is not performed (step S1306: No),the imaging apparatus 100 returns to step S1301. By contrast with this,when the operation to finish playing the image is performed (step S1306:Yes), the imaging apparatus 100 returns to the main routine illustratedin FIG. 32.

A case (step S1303: No) will be described where, in step S1303, aselected image is not a video. In this case, the display control unit110 d controls the display unit 7 to display the selected image on thefull screen (step S1307), and the control unit 110 decides whether ornot an operation to switch the image is performed (step S1308). Morespecifically, when the special effect input switch 6 d is operated, thecontrol unit 110 decides whether or not a command signal for commandingto switch an image displayed on the display unit 7 is inputted from thespecial effect input switch 6 d. When the operation to switch the imageis performed (step S1308: Yes), the display control unit 110 d controlsthe display unit 7 to switch the image currently displayed on thedisplay unit 7 (step S1309), and the imaging apparatus 100 returns tostep S1307. By contrast with this, when the operation to switch theimage is not performed after the image is displayed on the full screenand a predetermined time, for example, 30 seconds passes (step S1308:No), the imaging apparatus 100 goes to step S1306.

With the third embodiment described above, the display control unit 110d controls the display unit 7 to display an image corresponding to imagedata generated at a frame rate of a shooting mode set by the shootingmode setting unit 110 b, and controls the display unit 7 to displayinformation related to a special effect which the special effect imagegenerating unit 110 c needs to apply to image data, in order from thehighest advisability corresponding to a combination with the frame rateof the shooting mode set by the shooting mode setting unit 110 b,referring to the advisability information table T11 stored in theadvisability information storing unit 109 d. By this means, the imagingapparatus 100 can present, to the user, information which serves as aguide for an optimal combination of a special effect for an imagegenerated at the set frame rate of the shooting mode.

Further, with the third embodiment, the display control unit 110 dcontrols the display unit 7 to first display icons of special effects ofthe highest advisability, so that the user can apply a special effect toan image generated at the set frame rate of the shooting mode withoutmissing a timing to capture an image of the subject due to a user'soperation of selecting the special effect.

In addition, although, with the third embodiment, the control unit 110selects and determines an icon which the user desires, according to asignal received by the touch panel 8 as input when the user touchesvarious icons displayed on the image on the display unit 7, the controlunit 110 may select and determine an icon which the user desires,according to a command signal which is received as input from thespecial effect input switch 6 d.

Consequently, by operating the special effect input switch 6 d, the usercan select and determine information related to a shooting mode andinformation related to a special effect displayed on the display unit 7.

Further, with the third embodiment, the display control unit 110 d maycontrol the display unit 7 to emphasize and display an image generatedbased on the shooting mode set by the shooting mode setting unit 110 b,according to a display mode of a changed color and/or size of a specialeffect icon, in order from the highest advisability of a combinationwith a special effect, referring to the advisability information tableT11 stored in the advisability information storing unit 109 d. By thismeans, the user can more intuitively select an optimal combination of ashooting mode and special effect, based on information which serves as aguide to select a combination of a shooting mode and special effect.

Further, although, with the third embodiment, the display control unit110 d controls the display unit 7 to display only two special effecticons, the display control unit 110 d may control the display unit 7 todisplay special effect icons corresponding to special effects of thesame advisability. In this case, the display control unit 110 d maycontrol the display unit 7 to display reduced special effect icons inone vertical line or one horizontal line.

Further, with the third embodiment, by connecting the communication unit10 to an external processing device such as a personal computer orserver, the control unit 110 may update or rewrite the advisabilityinformation table T11 stored in the advisability information storingunit 109 d based on information received through the communication unit10. By this means, the display control unit 110 d can display a specialeffect icon according to the advisability which takes into account acombination of a newly added special effect and a frame rate of ashooting mode. Further, by storing the advisability of a combination ofa newly added special effect and shooting mode in, for example, astorage medium such as a memory card which is attachable to the storingunit 109, the control unit 110 may acquire the advisability from thestorage medium and store the advisability in the advisabilityinformation storing unit 109 to update or rewrite the advisabilityinformation table T11.

Further, although, with the third embodiment, the posture detecting unit3 detects the posture state of the imaging apparatus 1, by, for example,detecting the acceleration produced when the user taps the displayscreen of the display unit 7, the posture detecting unit 3 may receivean operation signal of a touch operation to switch various shootingmodes and various settings of the imaging apparatus 100, and output thisoperation signal to the control unit 110.

Further, although, with the third embodiment, when the user touches theswitch icon Q113 or Q114, the display control unit 110 d switches aspecial effect icon, the display control unit 110 d may switch a specialeffect icon according to a trajectory which the user touches on thetouch panel 8 in an area of a special effect icon displayed on an imageon the display unit 7.

Fourth Embodiment

Next, a fourth embodiment will be described. The imaging apparatusaccording to the fourth embodiment differs from the above embodiments inconfigurations of a control unit and storing unit. Further, the imagingapparatus according to the fourth embodiment differs from the imagingapparatus according to the above embodiments in the operation. Hence,hereinafter, the configurations of the storing unit and control unit ofthe imaging apparatus according to the fourth embodiment will bedescribed, and then the operation of the imaging apparatus according tothe fourth embodiment will be described. In addition, the samecomponents will be assigned the same reference numerals in the drawings.

FIG. 41 is a block diagram illustrating a configuration of the imagingapparatus according to the fourth embodiment. As illustrated in FIG. 41,an imaging apparatus 200 includes the imaging unit 2, the posturedetecting unit 3, the light emitting unit 4, the clock 5, the operationinput unit 6, the display unit 7, the touch panel 8, the communicationunit 10, a storing unit 209 and a control unit 211.

The storing unit 209 is provided inside the imaging apparatus 200 in afixed manner, and is formed using a semiconductor memory such as a flashmemory or RAM. The storing unit 209 includes an image data storing unit209 a which stores image data captured by the imaging unit 2; a programstoring unit 209 b which stores various programs executed by the imagingapparatus 200 and capturing programs according to one embodiment; anadvisability information storing unit 209 c which stores an advisabilityinformation table which, when the special effect (art filter) is appliedto an image, assigns an advisability corresponding to a combination of aspecial effect and shooting mode; and a use history information storingunit 209 d which stores use history information showing a use history ofa combination of a special effect and shooting mode. In addition, thestoring unit 209 may have a function of a storage medium interface whichreads information stored in the storage medium while storing theinformation in a storage medium such as a memory card attached from anoutside.

The control unit 211 is formed with, for example, a CPU. The controlunit 211 reads a program from the program storing unit 209 b of thestoring unit 209 according to, for example, an operation signal from theoperation input unit 6 to execute, and performs overall control of theoperation of the imaging apparatus 200 by, for example, commanding eachunit forming the imaging apparatus 200 or transferring data. The controlunit 211 has an image processing unit 211 a, a shooting mode settingunit 211 b, a special effect image generating unit 211 c, a use historyupdating unit 211 d, an advisability changing unit 211 e, aninitializing unit 211 f and a display control unit 211 g.

The image processing unit 211 a applies various image processings toimage data outputted from the signal processing unit 2 h to output tothe storing unit 209. More specifically, the image processing unit 211 aapplies processings such as edge enhancement, color correction and γcorrection to image data outputted from the signal processing unit 2 h.

The shooting mode setting unit 211 b sets in the imaging apparatus 200one of a plurality of shooting modes for which a capturing conditionsuitable for each capturing scene is set in advance. More specifically,the shooting mode setting unit 211 b sets the shooting mode of theimaging apparatus 200 based on an operation signal inputted from, forexample, the mode switch 6 c and the menu switch 6 e, or characteristicsof image data generated by the imaging unit 2. Meanwhile, the capturingcondition is parameter information including, for example, an aperturevalue, shutter speed, ISO sensitivity, white balance and saturation.Further, characteristics of image data include, for example, contrast,saturation and an occupation area which a main subject occupies in animage included image data.

The special effect image generating unit 211 c generates a specialeffect image by applying special effect corresponding to informationreceived by the touch panel 8 as input, to an image corresponding toimage data generated by the imaging unit 2. More specifically, thespecial effect image generating unit 211 c generates a special effectimage by applying to image data generated by the imaging unit 2 aspecial effect obtained by using various optical filters such as a colorconversion filter or contrast adjustment filter in the front surface ofthe lens unit 2 a of the imaging unit 2, or the same effect as a specialeffect obtained by using a shooting lens for a specific use such as afish-eye lens or a tilt-shift mechanism attached shooting lens. Forexample, the special effect image generating unit 211 c applies, toimage data, processing including one of a plurality of imageprocessings, tone curve processing, blurring process, shading addingprocess, image synthesizing process, noise superimposing process, andsaturation adjusting process. In addition, the special effect imagegenerating unit 211 c may generate a special effect image by applyingtwo or more special effects of different effects to image data generatedby the imaging unit 2. Further, the special effect image generating unit211 c may continuously generate special effect images by sequentiallyapplying special effects to image data continuously generated by theimaging unit 2.

The use history updating unit 211 d updates use history informationstored in the use history information storing unit 209 d every time thespecial effect image generating unit 211 c generates a special effectimage.

The advisability changing unit 211 e changes the advisability of acombination used a number of times more than a predetermined number oftimes among combinations of special effects and shooting modes includedin the advisability information table stored in the advisabilityinformation storing unit 209 c based on use history information storedin the use history information storing unit 209 d.

The initializing unit 211 f initializes the advisability of theadvisability information table changed by the advisability changing unit211 e. More specifically, the initializing unit 211 f initializes theadvisability information table set upon shipping of the imagingapparatus 200 from a manufacturer, according to an operation signal fromthe menu switch 6 e.

The display control unit 211 g controls the display unit 7 to performdisplay. The display control unit 211 g controls the display unit 7 todisplay an image corresponding to image data generated by the imagingunit 2 according to the shooting mode set by the shooting mode settingunit 211 b, and controls the display unit 7 to display informationrelated to a special effect which the special effect image generatingunit 211 c needs to apply to an image, in order from the highestadvisability corresponding to a combination with a shooting mode,referring to the advisability information table stored in theadvisability information storing unit 209 c. The display control unit211 g controls the display unit 7 to display information related tospecial effects by means of icons. More specifically, the displaycontrol unit 211 g controls the display unit 7 to display special effecticons corresponding to special effects in order from the highestadvisability corresponding to the combination with the shooting mode,referring to advisability information table.

Processing performed by the imaging apparatus 200 employing the aboveconfiguration will be described. FIG. 42 is a flowchart illustrating anoutline of processing performed by the imaging apparatus 200. Inaddition, although an example will be described below where a stillimage is captured, the imaging apparatus 200 is applicable to videocapturing.

In FIG. 42, the control unit 211 first decides whether or not the powersource of the imaging apparatus 200 is turned on (step S2101). When thepower source of the imaging apparatus 200 is turned on (step S2101:Yes), the imaging apparatus 200 goes to step S2102. By contrast withthis, when the power source of the imaging apparatus 200 is not turnedon (step S2101: No), the imaging apparatus 200 finishes the processing.

Subsequently, the control unit 211 decides whether or not the imagingapparatus 200 is set to a shooting mode (step S2102). When the imagingapparatus 200 is set to the shooting mode (step S2102: Yes), the imagingapparatus 200 goes to step S2103 (described below). By contrast withthis, when the imaging apparatus 200 is not set to the shooting mode(step S2102: No), the imaging apparatus 200 goes to step S2123(described below).

A case (step S2102: Yes) will be described where, in step S2102, theimaging apparatus 200 is set to the shooting mode. In this case, thedisplay control unit 211 g controls the display unit 7 to display liveview images of an image corresponding to image data continuouslygenerated by the imaging unit 2 at constant and very short timeintervals (step S2103). As illustrated in FIG. 33, the camera mancaptures a subject O1 while checking a live view image such as the imageW_(n) displayed by, for example, the display unit 7.

In step S2104, the control unit 211 decides whether or not a releasesignal for commanding to capture an image is inputted from the releaseswitch 6 b. When a release signal for commanding to capture an image isnot inputted (step S2104: No), the control unit 211 decides whether ornot a command signal for commanding to display an operation menu isinputted from the menu switch 6 e when the user operates the menu switch6 e (step S2105). When receiving no command signal for commanding todisplay an operation menu as input from the menu switch 6 e (step S2105:No), the imaging apparatus 200 goes to step S2101. By contrast withthis, when receiving a command signal for commanding to display anoperation menu as input from the menu switch 6 e (step S2105: Yes), theimaging apparatus 200 goes to step S2106.

Subsequently, the display control unit 211 g controls the display unit 7to display the operation menu on the image W_(n) displayed on thedisplay unit 7 (step S2106). More specifically, as illustrated in FIG.43, the display control unit 211 g displays a shooting mode menu iconQ201, a special effect menu icon Q202, a strobe light menu icon Q203 andan end icon Q204 on the image Wn.

Hereinafter, various icons which the display control unit 211 g controlsthe display unit 7 to display illustrated in FIG. 43 will be described.The shooting mode menu icon Q201 receives an input of a command signalfor controlling the display unit 7 to display information related to aplurality of shooting modes for which a capturing condition suitable foreach capturing scene is set in advance. The special effect menu iconQ202 receives an input of a command signal for controlling the displayunit 7 to display information related to a plurality of special effectswhich are applicable to image data generated by the imaging unit 2. Thestrobe light menu icon Q203 receives an input of a command signal forcommanding the display unit 7 to display detailed parameters of a strobelight operation condition of strobe light (flash). As the detailedparameters of the strobe light operation condition, parameters such asautomatic light emission, light emission prohibition, red-eye lightemission and forced light emission are displayed. The end icon Q204receives an input of a command signal for finishing displaying anoperation menu.

After the display control unit 211 g controls the display unit 7 todisplay the operation menu on the image W_(n) displayed on the displayunit 7, the control unit 211 decides whether or not the shooting modemenu icon Q201 is selected (step S2107). More specifically, asillustrated in FIG. 43, when the user touches the shooting mode menuicon Q201 displayed on the image W_(n) on the display unit 7, thecontrol unit 211 decides whether or not the shooting mode menu icon Q201is selected according to the signal received by the touch panel 8 asinput. When the shooting mode menu icon Q201 is selected (step S2107:Yes), the imaging apparatus 200 goes to step S2108 (described below). Bycontrast with this, when the shooting mode menu icon Q201 is notselected (step S2107: No), the imaging apparatus 200 goes to step S2114(described below).

In step S2107, after the shooting mode menu icon Q201 is selected, thedisplay control unit 211 g controls the display unit 7 to display a listof shooting modes on the image W_(n) (step S2108). More specifically, asillustrated in FIG. 44, the display control unit 211 g controls thedisplay unit 7 to display on the image W_(n) displayed on the displayunit 7 a person shooting mode icon Q211, a sport shooting mode iconQ212, a scenery shooting mode icon Q213, a night scene shooting modeicon Q214, a macro shooting mode icon Q215, a night scene and personshooting mode icon Q216, and an automatic setting shooting mode iconQ217.

Hereinafter, each shooting mode illustrated in FIG. 44 will bedescribed.

With the person shooting mode, a capturing condition suitable to capturean image of a person is set.

With the sport shooting mode, a capturing condition suitable to capturean image of a moving subject is set.

With the scenery shooting mode, a capturing condition suitable tocapture an image of the scenery is set.

With the night scene shooting mode, the capturing condition suitable tocapture an image of a night scene is set.

With the macro shooting mode, a capturing condition suitable to performmacro (close-up) capturing is set.

With the night scene and person shooting mode, a capturing conditionsuitable to capture an image of a person in the night scene is set.

The automatic setting capturing refers to automatically deciding acapturing scene based on image data generated by the imaging unit 2, andautomatically setting a capturing condition suitable for the decidedcapturing scene.

The above person shooting mode, sport shooting mode, scenery shootingmode, night scene shooting mode, macro shooting mode and night scene andperson shooting mode are manually set by the user by deciding thecapturing scene.

Referring back to FIG. 42, in step S2109, the control unit 211 decideswhether or not a shooting mode is selected according to a manualsetting. More specifically, the control unit 211 decides which one ofthe person shooting mode icon Q211, sport shooting mode icon Q212,scenery shooting mode icon Q213, night scene shooting mode icon Q214,macro shooting mode icon Q215, and night scene and person shooting modeicon Q216, is selected. When the shooting mode icon is selectedaccording to the manual setting (step S2109: Yes), the shooting modesetting unit 211 b sets a shooting mode corresponding to the selectedshooting mode icon, in the imaging apparatus 200 (step S2110).

Subsequently, the display control unit 211 g controls the display unit 7to display the shooting mode icons corresponding to the shooting modeset by the shooting mode setting unit 211 b, on the image W_(n)displayed on the display unit 7 (step S2111), and the imaging apparatus200 returns to step S2101. More specifically, as illustrated in FIG. 45,the display control unit 211 g controls the display unit 7 to displaythe person icon Q221 corresponding to the person shooting mode, on theimage W_(n) displayed on the display unit 7.

A case (step S2109: No) will be described where, in step S2109, ashooting mode is selected by a manual setting. In this case, the controlunit 211 decides whether or not the automatic setting shooting mode iconQ217 which automatically decides the capturing scene is selected (stepS2112). When the automatic setting shooting mode icon Q217 is selected(step S2112: Yes), the imaging apparatus 200 goes to step S2110. Bycontrast with this, when the automatic setting shooting mode icon Q217is not selected (step S2112: No), the control unit 211 decides whetheror not a predetermined time, for example, 30 seconds passes after a listof shooting modes is displayed (step S2113). When a predetermined timedoes not pass after the list of the shooting modes is displayed (stepS2113: No), the imaging apparatus 200 returns to step S2109. By contrastwith this, when a predetermined time passes after the list of theshooting modes is displayed (step S2113: Yes), the imaging apparatus 200returns to step S2101.

A case (step S2107: No) will be described where, in step S2107, theshooting mode menu icon Q201 is not selected. In this case, the controlunit 211 decides whether or not the special effect menu icon Q202 isselected (step S2114). When the special effect menu icon Q202 is notselected (step S2114: No), the imaging apparatus 200 goes to step S2121(described below). By contrast with this, when the special effect menuicon Q202 is selected (step S2114: Yes), the imaging apparatus 200 goesto step S2115 (described below).

In step S2115, the display control unit 211 g controls the display unit7 to display information related to a special effect which needs to beapplied to an image displayed on the display unit 7, in order from theadvisability corresponding to a combination with the shooting mode setby the shooting mode setting unit 211 b, referring to the advisabilityinformation table stored in the advisability information storing unit209 c. More specifically, as illustrated in FIG. 46, the display controlunit 211 g controls the display unit 7 to display a fantastic focus iconQ231 and a rough monochrome icon Q232 on the image W_(n) displayed onthe display unit 7.

Hereinafter, a method of displaying special effect icons which thedisplay control unit 211 g controls the display unit 7 to display willbe described with reference to FIG. 47. FIG. 47 is a view illustratingan example of an advisability information table stored by theadvisability information storing unit 209 c.

As illustrated in FIG. 47, in the advisability table T21, (0 to 2) arewritten for advisabilities which match combinations of special effectswhich the special effect image generating unit 211 c can apply to imagedata and shooting modes. More specifically, when the shooting mode is“person shooting mode”, the advisability is “2” for a combination with“fantastic focus”, “rough monochrome”, “day dream”, “light tone”, “HDR”,“sketch” and “dramatic tone” of special effects. Further, theadvisability is “1” for the combination with “toy photo”, and theadvisability is “0” for the combination with “pop art”.

Hereinafter, each special effect and the advisability for a combinationof a shooting mode and a special effect illustrated in FIG. 47 will bedescribed.

First, each special effect illustrated in FIG. 47 will be described.

Pop art provides an effect of colorfully and vividly finishing acaptured image of vigorous colors.

Fantastic focus provides an effect of fantastically finishing a capturedimage unique to soft focus.

Rough monochrome provides an effect of finishing a captured image byemphasizing strength and roughness unique to a monochrome picture.

Toy photo provides an effect of finishing a captured image by dimmingthe surrounding of the image as if the image is captured by toy photo.

Day dream provides an effect of finishing a captured image of a tripfilled with blue soft light.

Light tone refers to special effect processing of expressing brightnessand darkness of an image softly.

HDR provides an effect of finishing a captured image with a dynamicrange of actual scenery.

Sketch provides an effect of finishing a captured image by drawing theprofile by line drawing and emphasizing the profile.

Dramatic tone provides an effect of finishing a captured image byshowing like a fiction an actual space by means of brightness anddarkness which are created from the change of a local contrast and whichare unrealistic.

Next, the advisability of a combination of a shooting mode and a specialeffect will be described. Pop art is unsuitable to capture an image of aperson because capturing is performed by increasing the saturation veryhigh. Further, pop art requires multiple portions having colors, andtherefore is unsuitable for the night scene. Hence, the advisability is“0” for the person shooting mode, and the advisability is “1” for thenight scene shooting mode and night scene and person shooting mode.

Fantastic focus is a little unsuitable to capture an image of a movingperson because a hard motion image and a fantastic image contradicts,thereby decreasing an effect. Hence, the advisability is “1” for thesport shooting mode.

Rough monochrome is a little unsuitable to capture the image of thenight scene because the night scene is captured by monochromatically.Hence, the advisability is “1” for the night scene shooting mode andnight scene and person shooting mode.

Toy photo is unsuitable to capture the image of the night scene becausethe surrounding portion is dimmed, thereby making it difficult torecognize whether or not the night scene is dimmed when the sky is darklike the night scene. Further, toy photo is a little unsuitable tocapture an image of a person or a moving person because the scenery isassumed. Hence, the advisability is “0” for the night scene shootingmode and night scene and person shooting mode, and the advisability is“1” for the person shooting mode and sport shooting mode.

Day dream is unsuitable to capture the image of the night scene becausethe black level of the pixels is decreased, thereby losing theatmosphere of the night scene. Hence, the advisability is “0” for thenight scene shooting mode, and the advisability is “1” for the nightscene and person shooting mode.

Light tone is a little unsuitable to capture an image of a night scenebecause brightness and darkness of the image are finished softly. Hence,the advisability is “1” for the night scene shooting mode and nightscene and person shooting mode.

HDR is unsuitable to capture an image of a moving subject or a closesubject because shaking needs to be prevented to overlay a plurality ofimages. Hence, the advisability is “0” for the sport shooting mode, andthe advisability is “1” for the macro shooting.

Sketch is unsuitable to capture an image of a night scene because theprofile is drawn by line drawing and emphasized. Further, sketch is alittle unsuitable to capture scenery because the scenery including, forexample, sky or snow becomes plain. Hence, the advisability is “0” forthe night shooting mode, and the advisability is “1” for the sceneryshooting mode and night scene and person shooting mode.

Dramatic tone is a little unsuitable to capture an image of a nightscene because shading is emphasized. Hence, the advisability is “1” forthe night scene shooting mode and night scene and person shooting mode.

Thus, with the advisabilities in the advisability information table T21,a higher numerical value is set stepwise for a better combination of aspecial effect and a shooting mode. Further, the advisability in theadvisability information table T21 is set to “0” for unsuitablecombinations of special effects and shooting modes. By this means, whenthe display control unit 211 g controls the display unit 7 to displayinformation related to a special effect as a special effect icon, thedisplay control unit 211 g controls the display unit 7 to display theinformation in order from the highest advisability of the combinationwith the shooting mode set by the shooting mode setting unit 211 b,referring to the advisability information table T21 stored in theadvisability information storing unit 209 c. For example, as illustratedin FIG. 46, when the display control unit 211 g controls the displayunit 7 to display the person icon Q221 indicating the person shootingmode, on the image W_(n) currently displayed on the display unit 7, thedisplay control unit 211 g controls the display unit 7 to display thefantastic focus icon Q231 and rough monochrome icon Q232 as specialeffect icons.

Further, the display control unit 211 g controls the display unit 7 todisplay only information related to a special effect which can be usedin combination with a shooting mode set by the shooting mode settingunit 211 b. For example, when the shooting mode setting unit 211 b setsthe person shooting mode, the display control unit 211 g controls thedisplay unit 7 to display only special effect icons related to fantasticfocus, rough monochrome, toy photo, day dream, light tone, HDR, sketchand dramatic tone which can be used in combination with the personshooting mode. As a result, the imaging apparatus 200 can present, tothe user, information which serves as a guide for an optimal combinationof special effects for image generated by the imaging unit 2 accordingthe set shooting mode. In addition, the numerical values illustrated inFIG. 47 are only examples and by no means limited to these, and anumerical value of the advisability may be changed by, for example, thespecial effect input switch 6 d. Further, a lower advisability may beset stepwise for a better combination of a special effect and a shootingmode.

Referring back to FIG. 42, in step S2116, the control unit 211 decideswhether or not a special effect icon is selected. More specifically, asillustrated in FIG. 46, the control unit 211 decides whether or not theuser touches the fantastic focus icon Q231 or rough monochrome icon Q232displayed on the screen of the display unit 7. When the special effecticon is selected (step S2116: Yes), the special effect image generatingunit 211 c generates a special effect image by applying a special effectsuch as fantastic focus corresponding to the special effect icon whichthe user touches, to image data currently displayed on the display unit7 (step S2117).

Subsequently, the display control unit 211 g controls the display unit 7to display the special effect image generated by the special effectimage generating unit 211 c (step S2118), and the imaging apparatus 200returns to step S2101. More specifically, as illustrated in FIG. 48, thedisplay control unit 211 g controls the display unit 7 to display aspecial effect image W_(m) (m=natural number) by applying a specialeffect such as fantastic focus, to the image by the special effect imagegenerating unit 211 c. By this means, live view images displayed on thedisplay unit 7 are applied special effects, so that the user can captureimages while checking a shooting mode and special effect. In addition,although, in FIG. 48, an image is expressed by dotted lines to show aspecial effect of fantastic focus, this image is only an example.

A case (step S2116: No) will be described where, in step S2116, the userdoes not select a special effect icon. In this case, the control unit211 decides whether or not the user performs an operation to switch aspecial effect icon (step S2119). More specifically, as illustrated inFIG. 46, the control unit 211 decides whether the user touches theswitch icon Q233 or switch icon Q234 which receives an input of acommand signal for commanding to switch to a special effect icon of thesecond highest advisability from the special effect icon currentlydisplayed on the display unit 7. When the user does not perform anoperation to switch the special effect icon within a predetermined time,for example, within 30 seconds (step S2119: No), the imaging apparatus200 returns to step S2101. By contrast with this, when the user performsan operation to switch a special effect icon (step S2119: Yes), theimaging apparatus 200 goes to step S2120.

Subsequently, the display control unit 211 g controls the display unit 7to display a special effect icon of the second highest advisability,referring to the advisability information table T21 (step S2120), andthe imaging apparatus 200 returns to step S2116. More specifically, asillustrated in FIG. 49, the display control unit 211 g controls thedisplay unit 7 to display the fantastic focus icon Q231, roughmonochrome icon Q232 to day dream icon Q241 and light tone icon Q242((a)→(b) of FIG. 49). By this means, the user can adequately select adesired special effect.

A case (step S2114: No) will be described where, in step S2114, theshooting mode menu icon Q201 and special effect menu icon Q202 are notselected. In this case, the control unit 211 decides whether or not theend icon Q204 is selected (step S2121). When the end icon Q204 is notselected (step S2121: No), the imaging apparatus 200 returns to stepS2106. By contrast with this, when the end icon Q204 is selected (stepS2121: Yes), the imaging apparatus 200 returns to step S2101.

A case (step S2104: Yes) will be described where, in step S2104, arelease signal for commanding to capture an image is inputted from therelease switch 6 b. In this case, the imaging apparatus 200 executescapturing process of capturing a subject (step S2122), and the imagingapparatus 200 returns to step S2101.

A case (step S2102: No) will be described where, in step S2102, theimaging apparatus 200 is not set to the shooting mode. In this case, theimaging apparatus 200 executes playback/display process of playingcaptured image data (step S2123), and the imaging apparatus 200 returnsto step S2101.

Next, capturing process in step S2122 illustrated in FIG. 42 will bedescribed. FIG. 50 is a flowchart illustrating an outline of capturingprocess.

As illustrated in FIG. 50, the imaging apparatus 200 captures thesubject (step S2201). In addition, this capturing includes a specialeffect image generated by applying a special effect from the specialeffect image generating unit 211 c, to an image corresponding to imagedata generated by the imaging unit 2 according to the shooting mode setby the shooting mode setting unit 211 b.

Subsequently, the control unit 211 generates an image file of thecaptured image data and controls the image data storing unit 209 a tostore the captured image data (step S2202). In case of a still image,this image file stores at least the image data, presence/absenceinformation of a special effect applied to the image data, typeinformation of the special effect applied to the image data, shootingmode information and date information. Further, in case of videocapturing, this image file stores at least image data, presence/absenceinformation of a special effect applied to the image data, typeinformation of the special effect, use count information of the specialeffect, shooting mode information and date information.

Then, the display control unit 211 g controls the display unit 7 toREC-view display an image corresponding to the captured image data onlyfor a predetermined time, for example, 30 seconds (step S2203).

After the display unit 7 REC-view displays the captured image, thecontrol unit 211 decides whether or not the REC-view displayed image isa special effect image (step S2204). More specifically, the control unit211 decides whether or not a special effect is applied to the REC-viewdisplayed image, based on information included in the image file of theREC-view displayed image. When a special effect is not applied to theREC-view displayed image (step S2204: No), the imaging apparatus 200returns to the main routine illustrated in FIG. 42. By contrast withthis, when a special effect is not applied to the REC-view displayedimage (step S2204: Yes), the imaging apparatus 200 goes to step S2205.

Subsequently, the use history updating unit 211 d updates a use historyof a combination of a special effect applied to the REC-view displayedimage and a shooting mode, for use history information stored in the usehistory information storing unit 209 d (step S2205).

Then, the advisability changing unit 211 e changes the advisability of acombination used a number of times more than a predetermined number oftimes among combinations of special effects and shooting modes includedin the advisability information table T21 stored in the advisabilityinformation storing unit 209 c based on use history information storedin the use history information storing unit 209 d (step S2206), and theimaging apparatus 200 returns to the main routine illustrated in FIG.42. More specifically, the advisability changing unit 211 e changes theadvisability from “1” to “2” for a combination used a number of timesmore than a predetermined number of times, for example, 100 times, forexample, a combination of “toy photo” and “person shooting mode” amongcombinations of special effects and shooting modes included in theadvisability information table T21 stored in the advisabilityinformation storing unit 209 c (see FIG. 47). By this means, the imagingapparatus 200 can present, to the user, information which serves as aguide for a combination of a special effect and shooting mode takinginto account a use situation of the user.

Next, playback/display process illustrated in FIG. 42 will be described.FIG. 51 is a flowchart illustrating an outline of playback/displayprocess illustrated in FIG. 42.

As illustrated in FIG. 51, the display control unit 211 g first controlsthe display unit 7 to display an image selection screen collectivelydisplaying a plurality of images stored in the image data storing unit209 a (step S2301).

Subsequently, the control unit 211 decides whether or not the usertouches the touch panel 8 to select an image on the image selectionscreen displayed on the display unit 7 (step S2302). When the user doesnot select an image from the image selection screen after apredetermined time, for example, 3 seconds passes (step S2302: No), theimaging apparatus 200 goes to step S2306 (described below). By contrastwith this, when the user selects an image on the image selection screen(step S2302: Yes), the display control unit 211 g controls the displayunit 7 to display the image selected by the user on the full screen(step S2303), and the imaging apparatus 200 goes to step S2304.

In step S2304, when the user operates the special effect input switch 6d, the control unit 211 decides whether or not a command signal forcommanding to switch an image displayed on the display unit 7 isinputted from the special effect input switch 6 d. When receiving acommand signal for commanding to switch the image as input (step S2304:Yes), the display control unit 211 g switches the image currentlydisplayed on the display unit 7 (step S2305), and the imaging apparatus200 returns to step S2303. By contrast with this, when a command signalfor commanding to switch the image is not inputted (step S2304: No), thecontrol unit 211 decides whether or not an operation to finish playingthe image is performed (step S2306). More specifically, when the useroperates the mode switch 6 c, the control unit 211 decides whether ornot the imaging apparatus 200 is switched from the playback mode to theshooting mode. When the operation to finish playing the image is notperformed (step S2306: No), the imaging apparatus 200 returns to stepS2301. By contrast with this, when the operation to finish playing theimage is performed (step S2306; Yes), the imaging apparatus 200 returnsto the main routine illustrated in FIG. 42.

With the fourth embodiment described above, the display control unit 211g controls the display unit 7 to display image data generated accordingto the shooting mode set by the shooting mode setting unit 211 b, andcontrols the display unit 7 to display information related to a specialeffect which the special effect image generating unit 211 c needs toapply to the image generated by the imaging unit 2, in order from thehighest advisability of a combination with the shooting mode set by theshooting mode setting unit 211 b, referring to the advisabilityinformation table T21 stored in the advisability information storingunit 209 c. By this means, the imaging apparatus 200 can present, to theuser, information which serves as a guide for a combination of a specialeffect for image data generated according to the set shooting mode.

Further, with the fourth embodiment, the display control unit 211 gcontrols the display unit 7 to first display icons of special effects ofthe highest advisability, so that the user can obtain a special effectimage by applying a special effect to image data generated at the setshooting mode without missing a timing to capture an image of thesubject due to an operation of selecting the special effect.

Furthermore, with the fourth embodiment, the advisability changing unit211 e changes the advisability of a combination used a number of timesmore than a predetermined number of times among combinations of specialeffects and shooting modes included in the advisability informationtable T21 stored in the advisability information storing unit 209 c,based on use history information stored in the use history informationstoring unit 209 d. By this means, the imaging apparatus 200 canpresent, to the user, information which serves as a guide for acombination of a special effect and shooting mode taking into account ause situation of the user.

In addition, although, with the fourth embodiment, the control unit 211selects and determines an icon which the user desires according to asignal received by the touch panel 8 as input when various iconsdisplayed on the image by the display unit 7 are touched, the controlunit 211 may select and determine an icon which the user desires,according to a command signal inputted from the special effect inputswitch 6 d. Consequently, by operating the special effect input switch 6d, the user can select and determine information related to a shootingmode and information related to a special effect displayed on thedisplay unit 7.

Further, with the fourth embodiment, the display control unit 211 g maycontrol the display unit 7 to display image data generated according tothe shooting mode set by the shooting mode setting unit 211 b, bychanging a color and/or size of a special effect icon, in order from thehighest advisability of a combination with a special effect, referringto the advisability information table T21 stored in the advisabilityinformation storing unit 209 c. By this means, the user can moreintuitively check information which serves as a guide to select acombination of a shooting mode and special effect.

Further, although, with the fourth embodiment, the display control unit211 g controls the display unit 7 to display only two special effecticons, the display control unit 211 g may control the display unit 7 todisplay special effect icons corresponding to special effects of thesame advisability. In this case, the display control unit 211 g maycontrol the display unit 7 to display reduced special effect icons inone vertical line or one horizontal line.

Further, with the fourth embodiment, when receiving a command signal forcommanding to display an operation menu as input from the menu switch 6e, the display control unit 211 g may control the display unit 7 toalign and display special effects in one vertical line in order from thehighest advisability of combinations with special effects for an imagegenerated according to a shooting mode set by the shooting mode settingunit 211 b, referring to the advisability information table T21 storedin the advisability information storing unit 209 c. More specifically,as illustrated in FIG. 52, when the user operates the special effectinput switch 6 d, the display control unit 211 g may control the displayunit 7 to display a list of the special effect menu M1 in case where acommand signal for commanding to select the special effect menu M1 isinputted from the special effect input switch 6 d. In this case, whenthe user operates the special effect input switch 6 d and selectsinitialization M2, the initializing unit 211 f initializes theadvisabilities of the advisability information table T21 changed by theadvisability changing unit 211 e. By this means, the imaging apparatus200 can have the initialized advisability information table T21 set uponshipping from a manufacturer.

Further, with the fourth embodiment, when the user inputs a commandsignal for commanding to display the advisability information table T21stored in the advisability information storing unit 209 c through themenu switch 6 e, the display control unit 211 g may control the displayunit 7 to display the advisability information table T21. Morespecifically, as illustrated in FIG. 53, the display control unit 211 gmay control the display unit 7 to display the advisability informationtable T21. In this case, the user may change the advisability written inthe advisability information table T21 by operating the special effectinput switch 6 d. By this means, the user can change the advisabilitywhile checking the advisability which serves as a guide for acombination of a shooting mode and special effect.

Further, the above method of displaying the advisability informationtable displayed on the display unit 7 may adopt another display method.As illustrated in FIG. 54, the display control unit 211 g may controlthe display unit 7 to display the advisability information table T22,and control the display unit 7 to display a color corresponding to theadvisability written in the advisability information table T22. By thismeans, the user can more intuitively check the advisability which servesas a guide for a combination of a shooting mode and special effect. Inaddition, in FIG. 54, the colors are expressed by patterns. Further,when the advisabilities are shown by colors, the display control unit211 g may control the display unit 7 to sequentially display thedistinctive colors in order from the highest advisability, for example,red, green and black in order. Further, the display control unit 211 gmay control the display unit 7 to display the advisabilities withpatterns.

Further, the above method of displaying the advisability informationtable displayed on the display unit 7 may adopt another display method.As illustrated in FIG. 55, the display control unit 211 g may controlthe display unit 7 to display an advisability information table T23, andmay control the display unit 7 to change and display the advisabilitieswritten in the advisability information table T23 to bar charts.Further, the display control unit 211 g may change lengths of bar chartsaccording to the number of times of use by the user. Furthermore, thedisplay control unit 211 g may control the display unit 7 to display thenumber of times of use by the user together with the bar charts. By thismeans, the user can more intuitively check the advisability which servesas a guide for a combination of a shooting mode and special effect.

Further, with the fourth embodiment, the advisability informationstoring unit 209 c may store a plurality of advisability informationtables. For example, as illustrated in FIG. 56, the advisabilityinformation storing unit 209 c may store an advisability informationtable T24. Special effects different from the advisability informationtable T21 are written in the advisability information table T24. Morespecifically, the special effects include shading, cross, frame, noise,flare, blur, white out, illumination and soap bubble. Hereinafter, eachspecial effect illustrated in FIG. 56 will be described. Shadingprovides an effect of making surrounding portions of end portions of animage black. Cross provides an effect of adding a star-shaped icon to animage. Frame provides an effect of adding a frame to an image. Noiseprovides an effect of adding noise to the entire image. Flare providesan effect of adding hatching to an image. Blur provides an effect ofblurring process to the entire image. White out provides an effect ofmaking surroundings of end portions of an image white. Illuminationprovides an effect of applying a color filter to the entire image. Soapbubble provides an effect of adding an icon of soap bubbles to an image.According to soap bubble, an imaging apparatus in which a plurality ofzoom optical systems of different zoom magnifications, and the imagingelement 2 g are built in may perform image processing of changing thesize of soap bubbles according to a zoom magnification or may performimage processing of adjusting the frame in the horizontal directionaccording to the detection result of the posture detecting unit 3.Further, types of special effects are by no means limited to the contentof the advisability information tables T21 and T24, and, for example,art, ball, color mask, cube, mirror, mosaic, sepia, black/white, wave,ball frame and balloon effect may be added. Further, when the displayunit 7 displays the advisability information table T21, if a commandsignal for commanding to switch to another advisability informationtable is inputted from the special effect input switch 6 d, the displaycontrol unit 211 g may control the display unit 7 to display theadvisability information table T24.

Further, with the fourth embodiment, when the display unit 7 displaysthe advisability information table T25, if the user touches the touchpanel 8, the display control unit 211 g may control the display unit 7to display the advisability information table T25 near the touched area.

FIG. 57 is a view describing enlarged display near an area of the touchpanel 8 which the user touches. As illustrated in FIG. 57, when an areaR1 on the advisability information table T25 displayed on the displayunit 7 is touched ((a) of FIG. 57), the display control unit 211 gcontrols the display unit 7 to enlarge and display the advisabilityinformation table T25 near the touched area R1 ((b) of FIG. 57).Further, the display control unit 211 g controls the display unit 7 todisplay icons related to special effects which can cause an auxiliaryvisual effect with respect to each special effect. More specifically, asillustrated in (b) of FIG. 57, when the shooting mode is “person” andthe special effect is “pop art”, “shading”, “cross” and “blur” which arespecial effects which can cause auxiliary visual effects are displayedas icons.

Then, by selecting and touching a desired icon ((c) of FIG. 57), theuser can make a setting to add a special effect corresponding to thetouched icon. Further, the display control unit 211 g may control thedisplay unit 7 to add and display colors of icons corresponding tocombinations of special effects and special effects which can causeauxiliary visual effects with respect to the above special effects.Furthermore, when the advisabilities are shown by colors, the displaycontrol unit 211 g may control the display unit 7 to sequentiallydisplay the distinctive colors in order from the highest advisability,for example, red, green and black in order. Further, the display controlunit 211 g may control the display unit 7 to display as icons onlyspecial effects which can cause auxiliary visual effects with respect tospecial effects. In addition, in FIG. 57, the colors are expressed bypatterns.

Further, with the fourth embodiment, by connecting the communicationunit 10 to an external processing device such as a personal computer orserver, the advisability changing unit 211 e may update or rewrite theadvisability information table T21 stored in the advisabilityinformation storing unit 209 c, based on information received throughthe communication unit 10. By this means, the display control unit 211 gcan display special effect icons according to the advisability whichtakes into account combinations of newly added special effects andshooting modes. Further, by storing the advisability of a combination ofa newly added special effect and shooting mode in, for example, astorage medium such as a memory card which is attachable to the storingunit 209, the control unit 211 may acquire the advisability of acombination of the newly added special effect and shooting mode from thestorage medium and the advisability changing unit 211 e may store theadvisability in the advisability information storing unit 209 c toupdate or rewrite the advisability information table.

Further, although, with the fourth embodiment, the posture detectingunit 3 detects the posture state of the imaging apparatus 200, theposture detecting unit 3 may receive an operation signal of a touchoperation of switching various shooting modes or various settings of theimaging apparatus 200 by detecting an acceleration produced when theuser taps the display screen of the display unit 7, and output thisoperation signal to the control unit 211.

Further, although, with the fourth embodiment, when the user touches theswitch icons Q233 and Q234, the display control unit 211 g switches aspecial effect icon, the display control unit 211 g may switch to aspecial effect icon according to a trajectory which the user touches onthe touch panel 8 in an area of a special effect icon displayed on animage on the display unit 7.

Other Embodiment

Further, although, with the above embodiments of the present invention,an image processing unit and control unit are integrally formed, theimage processing unit (image processing engine) may be providedindividually in the imaging apparatus, and the control unit maytransfer, for example, various commands and data to this imageprocessing unit. It naturally follows that two or more image processingunits may be provided in the imaging apparatus. Further, similarly, thespecial effect image generating unit may also be provided individuallyin the imaging apparatus. Further, the special effect image generatingunit, image synthesizing unit and image processing unit may beintegrally formed (image engine).

Further, although, with the above embodiments of the present invention,the imaging apparatus has been described as a digital camera, theimaging apparatus is applicable to various electronic devices havingcapturing functions such as single-lens digital still cameras, digitalvideo cameras, camera-equipped mobile telephones or mobilemulti-function terminals.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An imaging apparatus comprising: an imaging unitthat captures an image of a subject to generate electronic image data ofthe subject; a display unit that displays an image corresponding to theimage data; an image separating unit that separates a subject image anda background image from the image corresponding to the image data; aspecial effect input unit that receives information related to specialeffects respectively applied to the subject image and the backgroundimage separated by the image separating unit; a special effect imagegenerating unit that generates a special effect image for each of thesubject image and the background image by applying special effectscorresponding to information received by the special effect input unitindividually to the subject image and the background image; a storingunit that stores special effect information for assigning anadvisability corresponding to a combination of special effects which thespecial effect image generating unit applies to the subject image andthe background image; a synthetic image generating unit that generates asynthetic image using the special effect images for the subject imageand the background image generated by the special effect imagegenerating unit; and a display control unit that controls the displayunit to display the image corresponding to the image data or thesynthetic image generated by the synthetic image generating unit,wherein the special effect image generating unit applies a specialeffect to one of the subject image and the background image to generatea special effect image, and when the synthetic image generating unitsynthesizes the special effect image with the other one of the subjectimage and the background image to generate a synthesized syntheticimage, the display control unit controls the display unit to display thesynthesized synthetic image, and controls the display unit to displayinformation related to special effects which need to be applied to theother one of the subject image and the background image, in order fromthe highest advisability corresponding to a combination with the specialeffect applied to the one of the subject image and the background image,referring to special effect information stored in the storing unit. 2.The imaging apparatus according to claim 1, further comprising a modeswitch that receives an input of a shooting mode, wherein the storingunit stores special effect information for assigning an advisabilitycorresponding to a combination of a shooting mode which can be set inthe imaging apparatus and a special effect applied to the subject imageor the background image, and when a shooting mode is set according to aninput from the shooting mode, the display control unit controls thedisplay unit to display information related to the special effects inorder from the highest advisability corresponding to a combination withthe set shooting mode, referring to the special effect informationstored in the storing unit.
 3. The imaging apparatus according to claim2, wherein the display control unit controls the display unit to displayonly information related to a special effect which can be used incombination with the special effect applied immediately before.
 4. Theimaging apparatus according to claim 3, further comprising a touch panelthat is provided on a display screen of the display unit and whichreceives a signal corresponding to a position of a contacting objectfrom an outside, wherein the display control unit controls the displayunit to display information related to the special effect by means of anicon, and the special effect image generating unit generates the specialeffect image by applying to the subject image or the background image, aspecial effect corresponding to the icon which the touch panel receivesas input in a contact area positioned on a display area of the icon. 5.The imaging apparatus according to claim 4, wherein the display controlunit controls the display unit to emphasize and display the iconaccording to a display mode of a changed color and/or size of the iconcorresponding to a combination with the special effect appliedimmediately before.
 6. An imaging method executed by an imagingapparatus that captures an image of a subject to generate electronicimage data of the subject, includes a display unit which displays animage corresponding to the generated image data of the subject, and thatcan apply a special effect to the image corresponding to the generatedimage data, the imaging method comprising: separating a subject imageand a background image from the image corresponding to the generatedimage data; generating a special effect image by applying a specialeffect to one of the separated subject image and background image;generating a synthetic image by synthesizing the other one of thesubject image and the background image, with the special effect image;and displaying the synthetic image, reading and referring to specialeffect information from a storing unit which stores the special effectinformation for assigning an advisability corresponding to a combinationof special effects applied to the subject image and the backgroundimage, and displaying information related to special effects which needto be applied to one of the subject image and the background image, inorder from the highest advisability corresponding to a combination withthe special effect applied to the one of the subject image and thebackground image.
 7. A non-transitory computer-readable recording mediumwith an executable program stored thereon, wherein the program instructsa processor of an imaging apparatus that captures an image of a subjectto generate electronic image data of the subject, includes a displayunit which displays an image corresponding to the generated image dataof the subject, and that can apply a special effect to the imagecorresponding to the generated image data, to perform: separating asubject image and a background image from the image corresponding to thegenerated image data; generating a special effect image by applying aspecial effect to one of the separated subject image and backgroundimage; generating a synthetic image by synthesizing the other one of thesubject image and the background image, with the special effect image;and displaying the synthetic image, reading and referring to specialeffect information from a storing unit which stores the special effectinformation for assigning an advisability corresponding to a combinationof special effects applied to the subject image and the backgroundimage, and displaying information related to special effects which needto be applied to one of the subject image and the background image, inorder from the highest advisability corresponding to a combination withthe special effect applied to the one of the subject image and thebackground image.